Jul 15 07:58:48.931914 ip-10-0-141-240 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 07:58:48.931928 ip-10-0-141-240 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 07:58:48.931937 ip-10-0-141-240 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 07:58:48.932246 ip-10-0-141-240 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 07:58:59.180962 ip-10-0-141-240 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 07:58:59.180982 ip-10-0-141-240 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f9b0fcd9d0d341b7a9b360d92bc914c0 -- Jul 15 08:00:35.224627 ip-10-0-141-240 systemd[1]: Starting Kubernetes Kubelet... Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.693674 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703490 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703506 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703511 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703514 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703517 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703520 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703523 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:35.724971 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703525 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703528 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703531 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703534 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703536 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703539 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703543 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703547 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703550 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703553 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703557 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703559 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703562 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703565 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703568 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703571 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703574 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703576 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703579 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.024151 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703582 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703584 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703587 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703591 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703594 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703598 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703602 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703604 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703607 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703613 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703617 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703621 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703624 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703626 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703630 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703632 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703636 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703640 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703643 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.029224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703646 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:35.771661 ip-10-0-141-240 systemd[1]: Started Kubernetes Kubelet. Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703649 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703651 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703655 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703658 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703661 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703668 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703670 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703673 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703675 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703678 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703681 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703683 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703686 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703688 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703691 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703693 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703696 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703699 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703701 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.031249 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703705 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703708 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703711 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703714 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703716 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703719 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703721 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703724 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703727 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703729 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703732 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703735 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703738 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703740 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703743 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703745 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703748 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703750 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703754 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703757 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.033920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703759 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703762 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703765 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703767 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703770 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703772 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703775 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703778 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703781 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703783 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703786 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703789 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703791 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.703795 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705535 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705544 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705548 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705552 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705555 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705558 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.035094 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705560 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705563 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705566 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705570 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705572 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705575 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705578 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705580 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705583 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705587 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705590 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705594 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705596 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705600 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705603 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705605 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705608 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705611 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705613 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.036193 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705616 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705619 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705622 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705624 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705630 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705633 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705636 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705639 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705642 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705645 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705649 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705651 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705654 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705657 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705661 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705664 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705667 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705670 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705673 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.037231 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705676 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705682 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705685 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705688 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705690 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705693 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705696 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705699 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705702 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705704 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705707 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705710 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705712 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705715 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705718 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705720 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705723 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705725 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705728 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705730 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.038028 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705733 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705736 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705738 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705741 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705745 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705747 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705750 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705752 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705755 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705757 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705760 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705763 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705765 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705769 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705772 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705774 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705777 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705780 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705783 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705786 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.038641 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705789 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705792 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705794 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705797 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705800 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705802 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705804 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705807 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705810 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705812 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705815 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705817 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705820 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705822 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.705825 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706202 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706208 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706212 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706215 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706217 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.039673 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706220 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706223 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706225 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706228 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706231 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706235 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706239 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706242 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706245 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706247 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706250 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706253 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706256 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706258 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706261 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706264 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706266 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706269 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706271 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.040280 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706274 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706278 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706281 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706284 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706286 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706289 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706292 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706294 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706317 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706320 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706323 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706325 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706329 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706331 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706334 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706336 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706339 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706341 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706345 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:36.040920 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706349 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706352 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706355 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706358 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706360 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706363 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706373 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706376 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706379 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706382 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706385 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706388 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706390 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706393 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706395 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706398 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706400 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706403 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706405 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706408 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.041486 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706410 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706413 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706415 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706418 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706420 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706423 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706425 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706428 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706430 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706433 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706435 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706439 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706443 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706446 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706449 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706451 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706454 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706457 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706461 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706466 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:36.042076 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706470 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706473 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706476 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706479 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706482 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706485 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706487 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706490 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706493 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706495 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706498 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706500 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706503 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706506 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706508 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.706510 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706581 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706588 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706596 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706602 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706609 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 08:00:36.042608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706612 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706616 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706621 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706626 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706630 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706633 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706636 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706639 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706642 2581 flags.go:64] FLAG: --cgroup-root="" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706645 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706648 2581 flags.go:64] FLAG: --client-ca-file="" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706652 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706655 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706660 2581 flags.go:64] FLAG: --cluster-domain="" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706663 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706666 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706669 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706673 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706677 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706679 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706682 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706686 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706689 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706692 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 08:00:36.043186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706694 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706698 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706701 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706705 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706708 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706711 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706714 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706717 2581 flags.go:64] FLAG: --enable-server="true" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706720 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706723 2581 flags.go:64] FLAG: --event-burst="100" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706727 2581 flags.go:64] FLAG: --event-qps="50" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706732 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706735 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706738 2581 flags.go:64] FLAG: --eviction-hard="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706742 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706745 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706748 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706751 2581 flags.go:64] FLAG: --eviction-soft="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706754 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706757 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706760 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706763 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706766 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706769 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706772 2581 flags.go:64] FLAG: --feature-gates="" Jul 15 08:00:36.043807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706776 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706779 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706782 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706785 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706788 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706791 2581 flags.go:64] FLAG: --help="false" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706794 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706797 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706800 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706802 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706806 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706809 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706812 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706814 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706817 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706820 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706823 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706827 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706831 2581 flags.go:64] FLAG: --kube-reserved="" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706834 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706837 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706840 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706843 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706846 2581 flags.go:64] FLAG: --lock-file="" Jul 15 08:00:36.044640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706848 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706851 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706854 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706860 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706863 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706866 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706869 2581 flags.go:64] FLAG: --logging-format="text" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706872 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706875 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706878 2581 flags.go:64] FLAG: --manifest-url="" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706881 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706885 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706888 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706892 2581 flags.go:64] FLAG: --max-pods="110" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706894 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706897 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706900 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706903 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706906 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706909 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706912 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706924 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706927 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706930 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 08:00:36.045252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706933 2581 flags.go:64] FLAG: --pod-cidr="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706937 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706943 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706946 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706949 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706952 2581 flags.go:64] FLAG: --port="10250" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706955 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706958 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e0c6058718b7a4ad" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706961 2581 flags.go:64] FLAG: --qos-reserved="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706964 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706968 2581 flags.go:64] FLAG: --register-node="true" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706971 2581 flags.go:64] FLAG: --register-with-taints="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706974 2581 flags.go:64] FLAG: --registry-burst="10" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706978 2581 flags.go:64] FLAG: --registry-qps="5" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706981 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706984 2581 flags.go:64] FLAG: --reserved-memory="" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706987 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706991 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706993 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706996 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.706999 2581 flags.go:64] FLAG: --runonce="false" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707002 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707005 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707008 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707011 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 08:00:36.045937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707013 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707017 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707020 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707024 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707026 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707029 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707032 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707035 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707038 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707043 2581 flags.go:64] FLAG: --system-cgroups="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707046 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707051 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707054 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707057 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707061 2581 flags.go:64] FLAG: --tls-min-version="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707064 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707067 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707070 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707073 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707076 2581 flags.go:64] FLAG: --v="2" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707080 2581 flags.go:64] FLAG: --version="false" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707084 2581 flags.go:64] FLAG: --vmodule="" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707088 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707091 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707212 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.046987 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707215 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707218 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707221 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707224 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707227 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707230 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707233 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707235 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707238 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707240 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707243 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707246 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707248 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707251 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707254 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707257 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707262 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707264 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707267 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707270 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.047707 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707272 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707275 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707277 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707280 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707283 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707286 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707288 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707291 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707310 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707317 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707320 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707323 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707326 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707329 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707332 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707335 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707338 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707340 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707343 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707346 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.174599 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707349 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707352 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707355 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707357 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707360 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707364 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707368 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707370 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707376 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707379 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707382 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707384 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707387 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707390 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707393 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707396 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707399 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707402 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707406 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.175201 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707409 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707412 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707414 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707417 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707420 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707422 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707425 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707427 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707430 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707432 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707435 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707437 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707439 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707442 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707444 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707447 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707449 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707452 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707455 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707457 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.175745 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707460 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707466 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707468 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707471 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707473 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707475 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707478 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707481 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707483 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707486 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707489 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707491 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707494 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707496 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707499 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707502 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707511 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707514 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.176308 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.707517 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.707525 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.714592 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.714609 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.714627 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.714633 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714690 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714695 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714699 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714702 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714705 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714708 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714711 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714714 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714717 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.176799 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714720 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714723 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714726 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714729 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714733 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714738 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714741 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714744 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714747 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714750 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714753 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714755 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714758 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714760 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714763 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714766 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714768 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714771 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714773 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714776 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.177250 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714784 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714788 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714791 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714794 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714797 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714800 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714803 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714805 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714808 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714811 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714813 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714816 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714818 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714821 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714823 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714828 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714831 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714833 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714836 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714839 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.177785 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714841 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714844 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714846 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714849 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714852 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714854 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714857 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714861 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714864 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714867 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714870 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714872 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714875 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714878 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714880 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714883 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714886 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714889 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714891 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714894 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.178389 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714897 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714899 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714902 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714905 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714908 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714910 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714913 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714917 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714919 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714922 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714924 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714927 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714929 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714932 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714935 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714937 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714940 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714943 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714945 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714948 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.178902 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714950 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714953 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714956 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714958 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714961 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714963 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714966 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714970 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714973 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.714975 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.714980 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715089 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715093 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715096 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715099 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 08:00:36.179462 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715102 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715105 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715107 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715110 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715113 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715117 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715120 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715123 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715126 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715129 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715132 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715134 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715137 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715140 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715143 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715146 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715149 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715152 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715154 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 08:00:36.179872 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715157 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715160 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715162 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715165 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715168 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715172 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715175 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715178 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715181 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715184 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715187 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715189 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715192 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715195 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715197 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715200 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715203 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715206 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715212 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 08:00:36.180418 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715215 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715218 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715221 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715224 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715226 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715229 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715232 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715235 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715239 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715241 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715244 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715246 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715249 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715252 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715254 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715257 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715259 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715262 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715265 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 08:00:36.180933 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715268 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715271 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715273 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715276 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715278 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715281 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715283 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715286 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715288 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715291 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715293 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715313 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715319 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715322 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715325 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715327 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715330 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715333 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715335 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715338 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 08:00:36.181477 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715340 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715344 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715348 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715351 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715354 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715357 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715360 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715363 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715366 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715369 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715372 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715376 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715378 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715382 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715384 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715388 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715390 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 08:00:36.181986 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:35.715393 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.715397 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.715515 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.719813 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.721428 2581 server.go:1019] "Starting client certificate rotation" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.721539 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.721572 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.724182 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.726511 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.738814 2581 log.go:25] "Validated CRI v1 runtime API" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.741744 2581 log.go:25] "Validated CRI v1 image API" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.742571 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 08:00:36.182498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.745316 2581 fs.go:135] Filesystem UUIDs: map[33dcdfdf-0730-4fde-8090-ef60b0e8390f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d2719e2d-bad9-48a1-9fbe-415acc292edf:/dev/nvme0n1p3] Jul 15 08:00:36.182838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.745332 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.750782 2581 manager.go:216] Machine: {Timestamp:2026-07-15 08:00:35.74915976 +0000 UTC m=+0.382622672 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099825 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29c220c88add89ec073d21d9dfc8f9 SystemUUID:ec29c220-c88a-dd89-ec07-3d21d9dfc8f9 BootID:f9b0fcd9-d0d3-41b7-a9b3-60d92bc914c0 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:dd:f9:5f:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:dd:f9:5f:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:a3:8d:a0:63:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.750917 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.751023 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.754464 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.754495 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.755391 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.755401 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 08:00:36.182942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.755413 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.755427 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.756643 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760213 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760363 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760389 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760399 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760419 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760430 2581 kubelet.go:398] "Adding apiserver pod source" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760440 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760812 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.760827 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.762137 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.763763 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.765540 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.765567 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766418 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766431 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766436 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766442 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766451 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766457 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766466 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766472 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766479 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766486 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766497 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.766513 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.767355 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.767362 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.770740 2581 server.go:1268] "Started kubelet" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.770823 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.771340 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.771408 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.775090 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.775898 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.777472 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.782201 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.782216 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.782422 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783082 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783103 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783212 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783286 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783293 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 15 08:00:36.183240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783401 2581 factory.go:55] Registering systemd factory Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783420 2581 factory.go:223] Registration of the systemd container factory successfully Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783449 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783659 2581 factory.go:153] Registering CRI-O factory Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783672 2581 factory.go:223] Registration of the crio container factory successfully Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783736 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783758 2581 factory.go:103] Registering Raw factory Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.783772 2581 manager.go:1196] Started watching for new ooms in manager Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.784089 2581 manager.go:349] Starting recovery of all containers Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.784209 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.784207 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.783616 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-240.ec2.internal.18c267b67705a6b7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-240.ec2.internal,UID:ip-10-0-141-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-240.ec2.internal,},FirstTimestamp:2026-07-15 08:00:35.770705591 +0000 UTC m=+0.404168480,LastTimestamp:2026-07-15 08:00:35.770705591 +0000 UTC m=+0.404168480,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-240.ec2.internal,}" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.790192 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.793743 2581 manager.go:354] Recovery completed Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.796758 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.797129 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.799311 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801143 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801175 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801188 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801801 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801814 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801814 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.801845 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.803332 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.803359 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.803373 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.804279 2581 policy_none.go:49] "None policy: Start" Jul 15 08:00:36.184435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.804292 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.804321 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.805337 2581 policy_none.go:47] "Start" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.811829 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-240.ec2.internal.18c267b678d65479 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-240.ec2.internal,UID:ip-10-0-141-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-240.ec2.internal,},FirstTimestamp:2026-07-15 08:00:35.801158777 +0000 UTC m=+0.434621669,LastTimestamp:2026-07-15 08:00:35.801158777 +0000 UTC m=+0.434621669,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-240.ec2.internal,}" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.820648 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-240.ec2.internal.18c267b678d6af35 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-240.ec2.internal,UID:ip-10-0-141-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-240.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-240.ec2.internal,},FirstTimestamp:2026-07-15 08:00:35.801182005 +0000 UTC m=+0.434644894,LastTimestamp:2026-07-15 08:00:35.801182005 +0000 UTC m=+0.434644894,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-240.ec2.internal,}" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.823371 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-240.ec2.internal.18c267b678d6d6c4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-240.ec2.internal,UID:ip-10-0-141-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-240.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-240.ec2.internal,},FirstTimestamp:2026-07-15 08:00:35.801192132 +0000 UTC m=+0.434655021,LastTimestamp:2026-07-15 08:00:35.801192132 +0000 UTC m=+0.434655021,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-240.ec2.internal,}" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.832189 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cdm8t" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.835216 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-240.ec2.internal.18c267b678d65479\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-240.ec2.internal.18c267b678d65479 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-240.ec2.internal,UID:ip-10-0-141-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-240.ec2.internal,},FirstTimestamp:2026-07-15 08:00:35.801158777 +0000 UTC m=+0.434621669,LastTimestamp:2026-07-15 08:00:35.803347432 +0000 UTC m=+0.436810322,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-240.ec2.internal,}" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.843893 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cdm8t" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874002 2581 manager.go:337] "Starting Device Plugin manager" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.874052 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874066 2581 server.go:87] "Starting device plugin registration server" Jul 15 08:00:36.187850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874391 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874404 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874530 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874625 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.874635 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.875163 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.875215 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.894711 2581 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2619 into the system container \"/system.slice\": " Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.974946 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.975938 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.975964 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.975977 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.976009 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:35.984773 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:35.984791 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-240.ec2.internal\": node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.001980 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.102398 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.102967 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.103002 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.103023 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.103064 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.105448 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.202618 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.203702 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal"] Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.203789 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.207485 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.207513 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.207527 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.208976 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209083 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209111 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209819 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209836 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209845 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.302687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209870 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209888 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.209897 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.210965 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.210985 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.211771 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientMemory" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.211794 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.211803 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeHasSufficientPID" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.239875 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-240.ec2.internal\" not found" node="ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.244039 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-240.ec2.internal\" not found" node="ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.285288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/869786661d939b62b49e72c2e18cdc59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-240.ec2.internal\" (UID: \"869786661d939b62b49e72c2e18cdc59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.285339 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.285403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.304573 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.303617 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.385984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/869786661d939b62b49e72c2e18cdc59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-240.ec2.internal\" (UID: \"869786661d939b62b49e72c2e18cdc59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.386016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.386039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.386081 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.386082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/869786661d939b62b49e72c2e18cdc59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-240.ec2.internal\" (UID: \"869786661d939b62b49e72c2e18cdc59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.393716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.386091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a73838634c65a1c0797837ab62c2d67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal\" (UID: \"0a73838634c65a1c0797837ab62c2d67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.497189 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.404086 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.507696 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.505086 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:36.605869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.542227 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.605869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:36.546685 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:36.605869 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:36.605686 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:37.866146 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:37.948558 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-240.ec2.internal\" not found" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:37.962695 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:37.962848 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:37.962875 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 08:00:37.993720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:37.979461 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.003606 2581 apiserver.go:52] "Watching apiserver" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.014849 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.015913 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-wp5tb","openshift-dns/node-resolver-xtzzx","openshift-image-registry/node-ca-zcxrr","openshift-multus/multus-additional-cni-plugins-pm62g","openshift-multus/multus-lpvls","openshift-network-diagnostics/network-check-target-nlrjg","kube-system/global-pull-secret-syncer-c2847","kube-system/konnectivity-agent-lcvbr","openshift-multus/network-metrics-daemon-4tcj5","openshift-network-operator/iptables-alerter-ckjsw","openshift-ovn-kubernetes/ovnkube-node-htgqh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc"] Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.017443 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.018561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.019806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.024851 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.025326 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.027959 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.027967 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.027979 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028079 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028210 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p8n4t\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028350 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028427 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r5k5p\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028461 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9tlf9\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.028523 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.029309 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.029724 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.029778 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-kubernetes\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-var-lib-kubelet\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-tmp-dir\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.125830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-cnibin\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030564 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030628 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030636 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030690 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdl5m\" (UniqueName: \"kubernetes.io/projected/77426b37-daba-4d8a-8319-9d1da9e75b37-kube-api-access-kdl5m\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030718 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-lib-modules\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ddf0841-975e-4676-a0c1-a7133fa9549d-serviceca\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030815 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030837 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hcvx2\"" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-systemd\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030886 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75fq2\" (UniqueName: \"kubernetes.io/projected/8425c324-9b90-4ddb-9025-00fb87f97531-kube-api-access-75fq2\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030907 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-modprobe-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030936 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030947 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysconfig\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.126783 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.030978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-tmp\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-hosts-file\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2vx4\" (UniqueName: \"kubernetes.io/projected/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-kube-api-access-s2vx4\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ddf0841-975e-4676-a0c1-a7133fa9549d-host\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031063 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-sys\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031081 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-system-cni-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031115 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-os-release\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-binary-copy\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031153 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-conf\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-run\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-tuned\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031228 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-host\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trnnv\" (UniqueName: \"kubernetes.io/projected/7ddf0841-975e-4676-a0c1-a7133fa9549d-kube-api-access-trnnv\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.127611 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.031453 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031849 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.031900 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8rllf\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.032651 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.032731 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.033863 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.035126 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.035191 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.036336 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.037598 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.039087 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041385 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041385 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041403 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041416 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tct8x\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041491 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041431 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041612 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal"] Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041699 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8hg6h\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041752 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.041874 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.042123 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043073 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043113 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043024 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tkszc\"" Jul 15 08:00:38.128231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043249 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tqfd4\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.043715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.044033 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.044061 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.044228 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.044384 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.047197 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.050024 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.050092 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal"] Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.062034 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-69mfj" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.071803 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-69mfj" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.090797 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 07:55:37 +0000 UTC" deadline="2027-12-31 18:48:52.867823583 +0000 UTC" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.090828 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12826h48m14.776997758s" Jul 15 08:00:38.129062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.126534 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysconfig\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-dbus\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2vx4\" (UniqueName: \"kubernetes.io/projected/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-kube-api-access-s2vx4\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-k8s-cni-cncf-io\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131956 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-sys\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-system-cni-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.131984 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysconfig\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-binary-copy\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132066 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-system-cni-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-conf\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132173 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-sys\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.228557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132236 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-netd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132272 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-conf\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-config\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-host\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trnnv\" (UniqueName: \"kubernetes.io/projected/7ddf0841-975e-4676-a0c1-a7133fa9549d-kube-api-access-trnnv\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132340 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-host\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-device-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-systemd-units\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d7ddd26-4029-46cd-89af-718eca5a1b89-iptables-alerter-script\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132537 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-os-release\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-cni-binary-copy\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-multus-daemon-config\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.229169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-etc-kubernetes\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-env-overrides\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgj9\" (UniqueName: \"kubernetes.io/projected/d6a9131f-37bd-480d-aa96-b226cef6d2c1-kube-api-access-mcgj9\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-socket-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-systemd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-log-socket\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-konnectivity-ca\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-cnibin\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdl5m\" (UniqueName: \"kubernetes.io/projected/77426b37-daba-4d8a-8319-9d1da9e75b37-kube-api-access-kdl5m\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132823 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-cnibin\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-kubelet\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-multus-certs\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-var-lib-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-ovn\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-sysctl-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-script-lib\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132967 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-lib-modules\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.132994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ddf0841-975e-4676-a0c1-a7133fa9549d-serviceca\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133053 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-lib-modules\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133089 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-kubelet-config\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75fq2\" (UniqueName: \"kubernetes.io/projected/8425c324-9b90-4ddb-9025-00fb87f97531-kube-api-access-75fq2\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-modprobe-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-tmp\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-netns\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-agent-certs\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.230816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-hosts-file\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ddf0841-975e-4676-a0c1-a7133fa9549d-host\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133251 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-registration-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133274 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ddf0841-975e-4676-a0c1-a7133fa9549d-host\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-hostroot\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133313 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-hosts-file\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-conf-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133352 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-etc-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-bin\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133397 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-modprobe-d\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-os-release\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-run\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-tuned\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133443 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-os-release\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.231922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-cnibin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-multus\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-netns\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133529 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-run\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xzsg\" (UniqueName: \"kubernetes.io/projected/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kube-api-access-9xzsg\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133560 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovn-node-metrics-cert\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133574 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133590 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pr24\" (UniqueName: \"kubernetes.io/projected/c7f9811f-5a14-4945-a92a-4cb1247a34f3-kube-api-access-6pr24\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d7ddd26-4029-46cd-89af-718eca5a1b89-host-slash\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133631 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-kubernetes\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133666 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-kubernetes\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-var-lib-kubelet\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.232803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133697 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ldc\" (UniqueName: \"kubernetes.io/projected/e3da972e-7da1-4a61-b795-a876dba7621b-kube-api-access-g2ldc\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-kubelet\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133754 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-slash\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133762 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-var-lib-kubelet\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133769 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-tmp-dir\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-node-log\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133820 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-bin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-systemd\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133864 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-sys-fs\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-system-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-socket-dir-parent\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133910 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5bp\" (UniqueName: \"kubernetes.io/projected/0d7ddd26-4029-46cd-89af-718eca5a1b89-kube-api-access-kb5bp\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.233849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.133978 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-systemd\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.134091 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8425c324-9b90-4ddb-9025-00fb87f97531-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.134118 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.134638 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ddf0841-975e-4676-a0c1-a7133fa9549d-serviceca\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.135224 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-tmp-dir\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.135395 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-binary-copy\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.135444 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.135845 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8425c324-9b90-4ddb-9025-00fb87f97531-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.137028 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-etc-tuned\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.137050 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77426b37-daba-4d8a-8319-9d1da9e75b37-tmp\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.152779 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2vx4\" (UniqueName: \"kubernetes.io/projected/bd80e0b2-f15c-41fa-96da-73cf4aba4e81-kube-api-access-s2vx4\") pod \"node-resolver-xtzzx\" (UID: \"bd80e0b2-f15c-41fa-96da-73cf4aba4e81\") " pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.152847 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdl5m\" (UniqueName: \"kubernetes.io/projected/77426b37-daba-4d8a-8319-9d1da9e75b37-kube-api-access-kdl5m\") pod \"tuned-wp5tb\" (UID: \"77426b37-daba-4d8a-8319-9d1da9e75b37\") " pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.152882 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75fq2\" (UniqueName: \"kubernetes.io/projected/8425c324-9b90-4ddb-9025-00fb87f97531-kube-api-access-75fq2\") pod \"multus-additional-cni-plugins-pm62g\" (UID: \"8425c324-9b90-4ddb-9025-00fb87f97531\") " pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.152905 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trnnv\" (UniqueName: \"kubernetes.io/projected/7ddf0841-975e-4676-a0c1-a7133fa9549d-kube-api-access-trnnv\") pod \"node-ca-zcxrr\" (UID: \"7ddf0841-975e-4676-a0c1-a7133fa9549d\") " pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-bin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.234745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234421 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-sys-fs\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-system-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-socket-dir-parent\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5bp\" (UniqueName: \"kubernetes.io/projected/0d7ddd26-4029-46cd-89af-718eca5a1b89-kube-api-access-kb5bp\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-dbus\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-k8s-cni-cncf-io\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234520 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-system-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-bin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234573 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-k8s-cni-cncf-io\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234516 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-sys-fs\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234588 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-socket-dir-parent\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234620 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234648 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-dbus\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.235999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234683 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-netd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-config\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-device-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234740 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-netd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-systemd-units\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234768 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-systemd-units\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-device-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d7ddd26-4029-46cd-89af-718eca5a1b89-iptables-alerter-script\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-os-release\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234858 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-cni-binary-copy\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-multus-daemon-config\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-etc-kubernetes\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.236869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-env-overrides\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgj9\" (UniqueName: \"kubernetes.io/projected/d6a9131f-37bd-480d-aa96-b226cef6d2c1-kube-api-access-mcgj9\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-socket-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.234990 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-systemd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235011 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-log-socket\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-konnectivity-ca\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-kubelet\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235088 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-multus-certs\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-var-lib-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-ovn\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235156 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-script-lib\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-kubelet-config\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235249 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.237737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-netns\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-agent-certs\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235307 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-cni-binary-copy\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-registration-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-os-release\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235412 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-hostroot\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-conf-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235444 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-cni-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-etc-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235467 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-bin\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-cnibin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-multus\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235513 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-netns\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.238197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235544 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235560 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xzsg\" (UniqueName: \"kubernetes.io/projected/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kube-api-access-9xzsg\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235574 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovn-node-metrics-cert\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pr24\" (UniqueName: \"kubernetes.io/projected/c7f9811f-5a14-4945-a92a-4cb1247a34f3-kube-api-access-6pr24\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235625 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d7ddd26-4029-46cd-89af-718eca5a1b89-host-slash\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ldc\" (UniqueName: \"kubernetes.io/projected/e3da972e-7da1-4a61-b795-a876dba7621b-kube-api-access-g2ldc\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235657 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-kubelet\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-slash\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-node-log\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235736 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-node-log\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235759 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-hostroot\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235779 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-multus-conf-dir\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235800 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-etc-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235819 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-cni-bin\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.239606 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235847 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3da972e-7da1-4a61-b795-a876dba7621b-multus-daemon-config\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235862 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-cnibin\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-run-netns\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235895 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-etc-kubernetes\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235907 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235926 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-cni-multus\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.235966 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.236033 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:38.736010075 +0000 UTC m=+2.127414680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236135 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d7ddd26-4029-46cd-89af-718eca5a1b89-iptables-alerter-script\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-env-overrides\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d7ddd26-4029-46cd-89af-718eca5a1b89-host-slash\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236504 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-kubelet\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.236552 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236569 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-socket-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.241311 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.236596 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:38.73658429 +0000 UTC m=+2.127988890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236615 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-host-slash\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236624 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-var-lib-kubelet\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236653 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-log-socket\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236669 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f6dd6fb1-a94d-4351-b84a-8550c44770d4-kubelet-config\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236708 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-systemd\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236800 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-etc-selinux\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.235254 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-config\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236870 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-netns\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236904 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovnkube-script-lib\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236943 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3da972e-7da1-4a61-b795-a876dba7621b-host-run-multus-certs\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236968 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-var-lib-openvswitch\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.236990 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6a9131f-37bd-480d-aa96-b226cef6d2c1-run-ovn\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.237021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-registration-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.237045 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.241946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.237052 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-konnectivity-ca\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.242589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.238477 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6a9131f-37bd-480d-aa96-b226cef6d2c1-ovn-node-metrics-cert\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.242589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.239127 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bf7d91e-5e46-4b77-8d49-1abf0ec5348b-agent-certs\") pod \"konnectivity-agent-lcvbr\" (UID: \"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b\") " pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.249323 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.249150 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:38.249323 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.249173 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:38.249323 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.249188 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:38.249323 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.249253 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:00:38.749237896 +0000 UTC m=+2.140642521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.251550 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pr24\" (UniqueName: \"kubernetes.io/projected/c7f9811f-5a14-4945-a92a-4cb1247a34f3-kube-api-access-6pr24\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.251657 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xzsg\" (UniqueName: \"kubernetes.io/projected/d38631e3-a116-4cbb-b197-0bf12a8cf6c5-kube-api-access-9xzsg\") pod \"aws-ebs-csi-driver-node-fz8sc\" (UID: \"d38631e3-a116-4cbb-b197-0bf12a8cf6c5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.251709 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ldc\" (UniqueName: \"kubernetes.io/projected/e3da972e-7da1-4a61-b795-a876dba7621b-kube-api-access-g2ldc\") pod \"multus-lpvls\" (UID: \"e3da972e-7da1-4a61-b795-a876dba7621b\") " pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.251743 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5bp\" (UniqueName: \"kubernetes.io/projected/0d7ddd26-4029-46cd-89af-718eca5a1b89-kube-api-access-kb5bp\") pod \"iptables-alerter-ckjsw\" (UID: \"0d7ddd26-4029-46cd-89af-718eca5a1b89\") " pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.252142 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgj9\" (UniqueName: \"kubernetes.io/projected/d6a9131f-37bd-480d-aa96-b226cef6d2c1-kube-api-access-mcgj9\") pod \"ovnkube-node-htgqh\" (UID: \"d6a9131f-37bd-480d-aa96-b226cef6d2c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.262399 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.267335 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" Jul 15 08:00:38.340140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.298469 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:38.436770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.350548 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" Jul 15 08:00:38.436770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.374087 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtzzx" Jul 15 08:00:38.436770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.385797 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zcxrr" Jul 15 08:00:38.436770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.395887 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pm62g" Jul 15 08:00:38.439944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.438928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lpvls" Jul 15 08:00:38.536233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.467885 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:00:38.536233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.497615 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ckjsw" Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.739880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.739918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.740028 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.740092 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:39.740073464 +0000 UTC m=+3.131478066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.740029 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:38.796142 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.740188 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:39.74016873 +0000 UTC m=+3.131573336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:38.814835 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.814810 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:38.841670 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.841638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:38.841772 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.841759 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:38.841812 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.841775 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:38.841812 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.841784 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:38.841872 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:38.841830 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:00:39.841815835 +0000 UTC m=+3.233220437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:38.843429 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:38.843404 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a9131f_37bd_480d_aa96_b226cef6d2c1.slice/crio-2410dccd41258f7b89f89bec12a90bdde23300382627b178f09668d3ddf33a1a WatchSource:0}: Error finding container 2410dccd41258f7b89f89bec12a90bdde23300382627b178f09668d3ddf33a1a: Status 404 returned error can't find the container with id 2410dccd41258f7b89f89bec12a90bdde23300382627b178f09668d3ddf33a1a Jul 15 08:00:38.848292 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:38.848263 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8425c324_9b90_4ddb_9025_00fb87f97531.slice/crio-fca7773f546e7dcdd7979baa8866851d2e3bb91be7a0cde1221e18afbcea353b WatchSource:0}: Error finding container fca7773f546e7dcdd7979baa8866851d2e3bb91be7a0cde1221e18afbcea353b: Status 404 returned error can't find the container with id fca7773f546e7dcdd7979baa8866851d2e3bb91be7a0cde1221e18afbcea353b Jul 15 08:00:38.850776 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:38.850731 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 08:00:38.867563 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:38.867543 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf7d91e_5e46_4b77_8d49_1abf0ec5348b.slice/crio-4924da129f0f967b04ee7e0f21ef8ca0ce1add1349d94f0e9885071120555b8c WatchSource:0}: Error finding container 4924da129f0f967b04ee7e0f21ef8ca0ce1add1349d94f0e9885071120555b8c: Status 404 returned error can't find the container with id 4924da129f0f967b04ee7e0f21ef8ca0ce1add1349d94f0e9885071120555b8c Jul 15 08:00:39.015903 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.015876 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7ddd26_4029_46cd_89af_718eca5a1b89.slice/crio-f5cbd93c340abfbb66346d887069d263e83af486283fded0ba4db6973d957dec WatchSource:0}: Error finding container f5cbd93c340abfbb66346d887069d263e83af486283fded0ba4db6973d957dec: Status 404 returned error can't find the container with id f5cbd93c340abfbb66346d887069d263e83af486283fded0ba4db6973d957dec Jul 15 08:00:39.040197 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.040166 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869786661d939b62b49e72c2e18cdc59.slice/crio-9e922603eacadaac71ed85a302d33f958cf448c813e85f914baa6a2f7a3a3e56 WatchSource:0}: Error finding container 9e922603eacadaac71ed85a302d33f958cf448c813e85f914baa6a2f7a3a3e56: Status 404 returned error can't find the container with id 9e922603eacadaac71ed85a302d33f958cf448c813e85f914baa6a2f7a3a3e56 Jul 15 08:00:39.074273 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.074228 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 07:55:38 +0000 UTC" deadline="2028-02-26 22:16:22.867473852 +0000 UTC" Jul 15 08:00:39.074273 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.074264 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14198h15m43.793212672s" Jul 15 08:00:39.132415 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.132385 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddf0841_975e_4676_a0c1_a7133fa9549d.slice/crio-c68ff4eac4925746e047333b1ce5c8dce0d1cbb0c3d845931a0cee1a1cdb2b46 WatchSource:0}: Error finding container c68ff4eac4925746e047333b1ce5c8dce0d1cbb0c3d845931a0cee1a1cdb2b46: Status 404 returned error can't find the container with id c68ff4eac4925746e047333b1ce5c8dce0d1cbb0c3d845931a0cee1a1cdb2b46 Jul 15 08:00:39.252548 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.252514 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3da972e_7da1_4a61_b795_a876dba7621b.slice/crio-c67b0d0500a42238eba39391e97b0e06f22774943a79d6ac93828da720dc6104 WatchSource:0}: Error finding container c67b0d0500a42238eba39391e97b0e06f22774943a79d6ac93828da720dc6104: Status 404 returned error can't find the container with id c67b0d0500a42238eba39391e97b0e06f22774943a79d6ac93828da720dc6104 Jul 15 08:00:39.345448 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.345373 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:39.345584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.345372 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:39.345584 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.345492 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:39.345584 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.345542 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:39.377046 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.376989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" event={"ID":"869786661d939b62b49e72c2e18cdc59","Type":"ContainerStarted","Data":"9e922603eacadaac71ed85a302d33f958cf448c813e85f914baa6a2f7a3a3e56"} Jul 15 08:00:39.377992 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.377970 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ckjsw" event={"ID":"0d7ddd26-4029-46cd-89af-718eca5a1b89","Type":"ContainerStarted","Data":"f5cbd93c340abfbb66346d887069d263e83af486283fded0ba4db6973d957dec"} Jul 15 08:00:39.379009 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.378989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lcvbr" event={"ID":"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b","Type":"ContainerStarted","Data":"4924da129f0f967b04ee7e0f21ef8ca0ce1add1349d94f0e9885071120555b8c"} Jul 15 08:00:39.380015 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.379997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerStarted","Data":"fca7773f546e7dcdd7979baa8866851d2e3bb91be7a0cde1221e18afbcea353b"} Jul 15 08:00:39.380990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.380972 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"2410dccd41258f7b89f89bec12a90bdde23300382627b178f09668d3ddf33a1a"} Jul 15 08:00:39.381906 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.381888 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lpvls" event={"ID":"e3da972e-7da1-4a61-b795-a876dba7621b","Type":"ContainerStarted","Data":"c67b0d0500a42238eba39391e97b0e06f22774943a79d6ac93828da720dc6104"} Jul 15 08:00:39.382838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.382822 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zcxrr" event={"ID":"7ddf0841-975e-4676-a0c1-a7133fa9549d","Type":"ContainerStarted","Data":"c68ff4eac4925746e047333b1ce5c8dce0d1cbb0c3d845931a0cee1a1cdb2b46"} Jul 15 08:00:39.567423 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.567387 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77426b37_daba_4d8a_8319_9d1da9e75b37.slice/crio-fea4d75a177346a6d54811784d2f82201879e5be7b71a2af0ad7ea0ffaa84e7c WatchSource:0}: Error finding container fea4d75a177346a6d54811784d2f82201879e5be7b71a2af0ad7ea0ffaa84e7c: Status 404 returned error can't find the container with id fea4d75a177346a6d54811784d2f82201879e5be7b71a2af0ad7ea0ffaa84e7c Jul 15 08:00:39.574937 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.574902 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd80e0b2_f15c_41fa_96da_73cf4aba4e81.slice/crio-68a2d7834912dcbdbaabf16f2622bb20231ba680597a0dff13e30cba31b174db WatchSource:0}: Error finding container 68a2d7834912dcbdbaabf16f2622bb20231ba680597a0dff13e30cba31b174db: Status 404 returned error can't find the container with id 68a2d7834912dcbdbaabf16f2622bb20231ba680597a0dff13e30cba31b174db Jul 15 08:00:39.594012 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.593612 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38631e3_a116_4cbb_b197_0bf12a8cf6c5.slice/crio-1414e35c29c81ffea324675c75cf7f6acad2f4b9ede838c8bee3cdb1834b7f34 WatchSource:0}: Error finding container 1414e35c29c81ffea324675c75cf7f6acad2f4b9ede838c8bee3cdb1834b7f34: Status 404 returned error can't find the container with id 1414e35c29c81ffea324675c75cf7f6acad2f4b9ede838c8bee3cdb1834b7f34 Jul 15 08:00:39.748921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.748879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:39.749106 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.748940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:39.749176 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.749132 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:39.749227 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.749174 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:39.749227 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.749216 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:41.749193858 +0000 UTC m=+5.140598463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:39.749335 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.749237 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:41.749227767 +0000 UTC m=+5.140632371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:39.849569 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:39.849491 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:39.849774 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.849731 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:39.849774 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.849751 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:39.849774 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.849764 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:39.849940 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:39.849820 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:00:41.849803717 +0000 UTC m=+5.241208321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:39.859723 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:00:39.859693 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a73838634c65a1c0797837ab62c2d67.slice/crio-f10de61f3e489dc062d18e78d40d38dd704b92a9a5335458515acf2cd3c7b624 WatchSource:0}: Error finding container f10de61f3e489dc062d18e78d40d38dd704b92a9a5335458515acf2cd3c7b624: Status 404 returned error can't find the container with id f10de61f3e489dc062d18e78d40d38dd704b92a9a5335458515acf2cd3c7b624 Jul 15 08:00:40.075313 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.075266 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 07:55:38 +0000 UTC" deadline="2028-04-24 12:42:22.093094262 +0000 UTC" Jul 15 08:00:40.075313 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.075312 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15580h41m42.017787232s" Jul 15 08:00:40.345990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.345463 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:40.345990 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:40.345586 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:40.392793 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.392732 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" event={"ID":"0a73838634c65a1c0797837ab62c2d67","Type":"ContainerStarted","Data":"f10de61f3e489dc062d18e78d40d38dd704b92a9a5335458515acf2cd3c7b624"} Jul 15 08:00:40.395125 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.395093 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" event={"ID":"d38631e3-a116-4cbb-b197-0bf12a8cf6c5","Type":"ContainerStarted","Data":"1414e35c29c81ffea324675c75cf7f6acad2f4b9ede838c8bee3cdb1834b7f34"} Jul 15 08:00:40.399590 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.399565 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtzzx" event={"ID":"bd80e0b2-f15c-41fa-96da-73cf4aba4e81","Type":"ContainerStarted","Data":"68a2d7834912dcbdbaabf16f2622bb20231ba680597a0dff13e30cba31b174db"} Jul 15 08:00:40.410050 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.410016 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" event={"ID":"77426b37-daba-4d8a-8319-9d1da9e75b37","Type":"ContainerStarted","Data":"fea4d75a177346a6d54811784d2f82201879e5be7b71a2af0ad7ea0ffaa84e7c"} Jul 15 08:00:40.443570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:40.443537 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 08:00:41.346331 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:41.346296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:41.346822 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.346465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:41.346888 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:41.346843 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:41.346961 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.346931 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:41.766568 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:41.766493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:41.766568 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:41.766542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:41.766794 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.766720 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:41.766794 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.766780 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:45.766757021 +0000 UTC m=+9.158161626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:41.767200 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.767173 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:41.767308 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.767251 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:45.76723499 +0000 UTC m=+9.158639597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:41.867195 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:41.867148 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:41.867391 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.867379 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:41.867461 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.867400 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:41.867461 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.867414 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:41.867553 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:41.867468 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:00:45.867449345 +0000 UTC m=+9.258853966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:42.345746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:42.345660 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:42.345892 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:42.345794 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:43.347622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:43.347557 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:43.348080 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:43.347685 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:43.348080 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:43.347960 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:43.348080 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:43.348067 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:44.346636 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:44.346210 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:44.346636 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:44.346314 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:45.346268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:45.346228 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:45.346268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:45.346275 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:45.346772 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.346379 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:45.346772 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.346474 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:45.803299 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:45.803219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:45.803299 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:45.803271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:45.803553 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.803460 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:45.803553 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.803475 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:45.803553 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.803521 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:53.803503709 +0000 UTC m=+17.194908310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:45.803553 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.803544 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:00:53.803535276 +0000 UTC m=+17.194939879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:45.903986 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:45.903948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:45.904183 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.904134 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:45.904183 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.904163 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:45.904183 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.904177 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:45.904342 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:45.904235 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:00:53.90421808 +0000 UTC m=+17.295622694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:46.346278 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:46.346226 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:46.346478 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:46.346382 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:47.350031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:47.348083 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:47.350031 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:47.348310 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:47.350031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:47.348738 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:47.350031 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:47.348887 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:48.346084 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:48.346052 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:48.346296 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:48.346198 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:49.346517 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:49.346024 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:49.346517 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:49.346154 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:49.347037 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:49.346643 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:49.347037 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:49.346743 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:50.345613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:50.345413 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:50.345784 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:50.345697 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:51.345598 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:51.345564 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:51.346043 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:51.345620 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:51.346043 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:51.345701 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:51.346043 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:51.345808 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:52.346264 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:52.346234 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:52.346695 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:52.346343 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:53.346320 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:53.346286 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:53.346723 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.346407 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:53.346723 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:53.346470 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:53.346723 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.346596 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:53.860657 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:53.860617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:53.860846 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:53.860668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:53.860846 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.860786 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:53.860846 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.860786 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:53.860980 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.860853 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:09.860832248 +0000 UTC m=+33.252236854 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:00:53.860980 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.860872 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:09.860863055 +0000 UTC m=+33.252267666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:00:53.961686 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:53.961652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:53.961862 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.961814 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:00:53.961862 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.961834 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:00:53.961862 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.961844 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:53.962003 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:53.961894 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:01:09.961878929 +0000 UTC m=+33.353283545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:00:54.346020 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:54.345940 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:54.346172 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:54.346082 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:55.345685 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:55.345649 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:55.346129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:55.345662 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:55.346129 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:55.345790 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:55.346129 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:55.345861 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:56.346095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:56.346065 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:56.346549 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:56.346162 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:57.346347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:57.346309 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:57.346765 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:57.346442 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:57.346765 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:57.346489 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:57.346765 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:57.346630 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:58.345559 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:58.345524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:00:58.345744 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:58.345643 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:00:59.346518 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.346295 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:00:59.346518 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.346333 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:00:59.346518 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:59.346465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:00:59.347099 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:00:59.346576 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:00:59.472701 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.472478 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" event={"ID":"869786661d939b62b49e72c2e18cdc59","Type":"ContainerStarted","Data":"b9a3d6403d540c145a73a4e1c184bf9b504334597db385efc947c39bf8df2419"} Jul 15 08:00:59.482880 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.482843 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:00:59.484522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.484438 2581 generic.go:356] "Generic (PLEG): container finished" podID="d6a9131f-37bd-480d-aa96-b226cef6d2c1" containerID="c9b677822d99dbfe5a0d13cc050d562592cd789980793c82fde83c596155e8aa" exitCode=1 Jul 15 08:00:59.484621 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.484535 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"91c029609b7c3436b755c07735468ed36ed0c3ef5bcac7baea5919b445af272e"} Jul 15 08:00:59.484621 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.484614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"ee24b8580bd3f8c74abe1d6644cfe8a5b0c81b8e524b70ed539672fc93f24c07"} Jul 15 08:00:59.484719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.484663 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerDied","Data":"c9b677822d99dbfe5a0d13cc050d562592cd789980793c82fde83c596155e8aa"} Jul 15 08:00:59.484719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.484712 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"5a0d272b854d835b9eda679b1bd027aac562f73616b0091d9355301fe6e48dcd"} Jul 15 08:00:59.489435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.488861 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-240.ec2.internal" podStartSLOduration=21.488847055 podStartE2EDuration="21.488847055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:00:59.488779988 +0000 UTC m=+22.880184611" watchObservedRunningTime="2026-07-15 08:00:59.488847055 +0000 UTC m=+22.880251678" Jul 15 08:00:59.497019 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.495485 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" event={"ID":"77426b37-daba-4d8a-8319-9d1da9e75b37","Type":"ContainerStarted","Data":"42cc8ea06fbc8ee24923ba148abfbb3995032fedfdc3e5b3e098596460c8a1b8"} Jul 15 08:00:59.515679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:00:59.515628 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wp5tb" podStartSLOduration=3.198316551 podStartE2EDuration="22.515606474s" totalImagesPullingTime="19.317289923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:00:59.515259649 +0000 UTC m=+22.906664298" watchObservedRunningTime="2026-07-15 08:00:59.515606474 +0000 UTC m=+22.907011098" Jul 15 08:01:00.346095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.345913 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:00.346254 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:00.346165 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:00.501589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.501508 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:01:00.502579 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.502534 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"bd16c12563f808139b59166a53ca62da6392c314e26113ee8f030b6a961eb999"} Jul 15 08:01:00.502685 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.502620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"0e8f922e1fc60a34dbd656029f69af0390bcbd1b8c149f0cad5e1ec61468bffd"} Jul 15 08:01:00.504564 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.504535 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtzzx" event={"ID":"bd80e0b2-f15c-41fa-96da-73cf4aba4e81","Type":"ContainerStarted","Data":"9e7287d4002900e35fcbb8dbd5e97d9e2edebfe6149ba0dde50d32819b450542"} Jul 15 08:01:00.506181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.506158 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lpvls" event={"ID":"e3da972e-7da1-4a61-b795-a876dba7621b","Type":"ContainerStarted","Data":"774aa651c3ddbc2f786118a34e81d339b98172ff0acc4564bbb8fa7bc242b5fa"} Jul 15 08:01:00.507749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.507724 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zcxrr" event={"ID":"7ddf0841-975e-4676-a0c1-a7133fa9549d","Type":"ContainerStarted","Data":"a61d23979fc0b7643bd51b134a1f65c7578586ac08d5bc3026ce149347cad4b3"} Jul 15 08:01:00.509263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.509237 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ckjsw" event={"ID":"0d7ddd26-4029-46cd-89af-718eca5a1b89","Type":"ContainerStarted","Data":"c44bcb8c3dd1a8a0b8b73fcf9c2c7dc8438e2d242dceda6566d897b8e90376b4"} Jul 15 08:01:00.510780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.510757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lcvbr" event={"ID":"5bf7d91e-5e46-4b77-8d49-1abf0ec5348b","Type":"ContainerStarted","Data":"4c22b2da1f4dfa2dcd4e8a8215d9c717a9c75dc8b636c7a9f9a6e349f7fa250f"} Jul 15 08:01:00.512411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.512387 2581 generic.go:356] "Generic (PLEG): container finished" podID="0a73838634c65a1c0797837ab62c2d67" containerID="9e1a7c4bd7c815a4d8104cc4bdb5f7096a3a30d1071935bad465e18db24349ce" exitCode=0 Jul 15 08:01:00.512512 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.512416 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" event={"ID":"0a73838634c65a1c0797837ab62c2d67","Type":"ContainerDied","Data":"9e1a7c4bd7c815a4d8104cc4bdb5f7096a3a30d1071935bad465e18db24349ce"} Jul 15 08:01:00.514122 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.514102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" event={"ID":"d38631e3-a116-4cbb-b197-0bf12a8cf6c5","Type":"ContainerStarted","Data":"52c6acc6b014335d4aa6a4727157fab81f38d2853661623632d509a0e44ad70d"} Jul 15 08:01:00.515726 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.515705 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="b50396d0fe981327e62e95363d8e94dcfee8a7963f84743a0144711feb44c49f" exitCode=0 Jul 15 08:01:00.515827 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.515788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"b50396d0fe981327e62e95363d8e94dcfee8a7963f84743a0144711feb44c49f"} Jul 15 08:01:00.551590 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.551553 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xtzzx" podStartSLOduration=4.277397686 podStartE2EDuration="23.55153888s" totalImagesPullingTime="19.274141194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:00.525136216 +0000 UTC m=+23.916540838" watchObservedRunningTime="2026-07-15 08:01:00.55153888 +0000 UTC m=+23.942943503" Jul 15 08:01:00.572371 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.572310 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lcvbr" podStartSLOduration=3.594167515 podStartE2EDuration="23.572290955s" totalImagesPullingTime="19.97812344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:00.571730868 +0000 UTC m=+23.963135512" watchObservedRunningTime="2026-07-15 08:01:00.572290955 +0000 UTC m=+23.963695578" Jul 15 08:01:00.607412 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.607371 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zcxrr" podStartSLOduration=3.890070776 podStartE2EDuration="23.607340603s" totalImagesPullingTime="19.717269827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:00.606950796 +0000 UTC m=+23.998355419" watchObservedRunningTime="2026-07-15 08:01:00.607340603 +0000 UTC m=+23.998745225" Jul 15 08:01:00.634191 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.634147 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lpvls" podStartSLOduration=3.486122516 podStartE2EDuration="23.634130211s" totalImagesPullingTime="20.148007695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:00.633942022 +0000 UTC m=+24.025346645" watchObservedRunningTime="2026-07-15 08:01:00.634130211 +0000 UTC m=+24.025534834" Jul 15 08:01:00.810490 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.810419 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:00.892008 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.891975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:00.997507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:00.997482 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:01.046984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.046958 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 08:01:01.097704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.097628 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:01.128010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.127878 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T08:01:01.046978217Z","UUID":"f0962990-efcc-4508-a44b-ffbe7bd98fc8","Handler":null,"Name":"","Endpoint":""} Jul 15 08:01:01.129348 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.129332 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 08:01:01.129467 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.129377 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 08:01:01.193517 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.193484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:01.345324 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.345285 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:01.345547 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:01.345418 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:01.345547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.345473 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:01.345636 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:01.345559 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:01.519292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.519213 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" event={"ID":"0a73838634c65a1c0797837ab62c2d67","Type":"ContainerStarted","Data":"d65a8efef543da37c79ea60aa2ce04f819739b5e296366e48df53984dd045624"} Jul 15 08:01:01.521140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.521111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" event={"ID":"d38631e3-a116-4cbb-b197-0bf12a8cf6c5","Type":"ContainerStarted","Data":"85cb4b3d711522edc1f68d143f3b72c05f7cb43f982d8aa912f2b2ba9324ad16"} Jul 15 08:01:01.538854 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.538813 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ckjsw" podStartSLOduration=4.665543814 podStartE2EDuration="24.538798089s" totalImagesPullingTime="19.873254275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:00.656171556 +0000 UTC m=+24.047576180" watchObservedRunningTime="2026-07-15 08:01:01.538798089 +0000 UTC m=+24.930202703" Jul 15 08:01:01.716466 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:01.716431 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:02.346254 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:02.346077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:02.346481 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:02.346344 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:02.528350 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:02.528329 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:01:02.528797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:02.528776 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"656517aad55b4695b57ce5a668883138eba293557ac36d1b6a72e933b309eb2d"} Jul 15 08:01:02.536366 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:02.536330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:02.637322 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:02.637221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:03.346369 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:03.346317 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:03.346560 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:03.346457 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:03.346560 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:03.346533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:03.346682 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:03.346635 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:03.534175 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:03.534135 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" event={"ID":"d38631e3-a116-4cbb-b197-0bf12a8cf6c5","Type":"ContainerStarted","Data":"04e604dfa772fb5bad566335f771679dd2311c824035181e8b3a6f20c9cbcc72"} Jul 15 08:01:03.556522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:03.556468 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fz8sc" podStartSLOduration=4.328694427 podStartE2EDuration="26.556446448s" totalImagesPullingTime="22.227752021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:03.556105163 +0000 UTC m=+26.947509787" watchObservedRunningTime="2026-07-15 08:01:03.556446448 +0000 UTC m=+26.947851080" Jul 15 08:01:03.556891 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:03.556867 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-240.ec2.internal" podStartSLOduration=25.556858726 podStartE2EDuration="25.556858726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:01.539615555 +0000 UTC m=+24.931020177" watchObservedRunningTime="2026-07-15 08:01:03.556858726 +0000 UTC m=+26.948263349" Jul 15 08:01:04.272210 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.271892 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:01:04.272501 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.272479 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:01:04.345638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.345607 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:04.345781 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:04.345718 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:04.540429 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.540342 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:01:04.541113 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.540781 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"379cadc8cf1f25a01575a3758e7fa1b3a3a31fc8d8522d99316ed856c900d3ba"} Jul 15 08:01:04.541303 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.541185 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:01:04.541585 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.541569 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lcvbr" Jul 15 08:01:04.541585 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.541583 2581 scope.go:117] "RemoveContainer" containerID="c9b677822d99dbfe5a0d13cc050d562592cd789980793c82fde83c596155e8aa" Jul 15 08:01:04.652952 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:04.652920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" limit=1024 Jul 15 08:01:05.345870 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:05.345840 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:05.346046 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:05.345964 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:05.346046 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:05.346006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:05.346161 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:05.346091 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:06.203397 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.203207 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nlrjg"] Jul 15 08:01:06.203805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.203462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:06.203805 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:06.203551 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:06.207139 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.207114 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c2847"] Jul 15 08:01:06.207250 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.207194 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:06.207302 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:06.207273 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:06.207903 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.207872 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tcj5"] Jul 15 08:01:06.208035 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.207951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:06.208035 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:06.208025 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:06.546879 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.546846 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="e163e5677b614433866f9eb00aec0132660bc7f26fd476129013e35e96c1a1b9" exitCode=0 Jul 15 08:01:06.547065 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.546929 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"e163e5677b614433866f9eb00aec0132660bc7f26fd476129013e35e96c1a1b9"} Jul 15 08:01:06.550770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.550752 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:01:06.551107 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.551087 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" event={"ID":"d6a9131f-37bd-480d-aa96-b226cef6d2c1","Type":"ContainerStarted","Data":"b4e40d149ebc61f7b8ee2b06024f0cc6aa5f3111f2d2b3cb477ed9e79799fb22"} Jul 15 08:01:06.552733 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.552069 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:06.552733 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.552421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:06.555390 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.552977 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:06.567282 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.567262 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:06.568072 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.568059 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:06.646140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:06.646098 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" podStartSLOduration=9.580501401 podStartE2EDuration="29.64608078s" totalImagesPullingTime="20.065579379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:06.64601645 +0000 UTC m=+30.037421075" watchObservedRunningTime="2026-07-15 08:01:06.64608078 +0000 UTC m=+30.037485402" Jul 15 08:01:07.346864 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:07.346831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:07.347346 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:07.346938 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:08.345259 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:08.345239 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:08.345394 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:08.345350 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:08.345394 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:08.345383 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:08.345563 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:08.345488 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:08.561331 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:08.561297 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="1968cdfd44eb5260d42e703329aac4dd96e8b3d17c1465df79e5c19a43390d04" exitCode=0 Jul 15 08:01:08.561849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:08.561395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"1968cdfd44eb5260d42e703329aac4dd96e8b3d17c1465df79e5c19a43390d04"} Jul 15 08:01:08.571799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:08.571767 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" podUID="d6a9131f-37bd-480d-aa96-b226cef6d2c1" containerName="ovnkube-controller" probeResult="failure" output="" Jul 15 08:01:09.345558 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:09.345534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:09.345724 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.345643 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:09.874895 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:09.874866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:09.874895 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:09.874901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:09.875466 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.875025 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:01:09.875466 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.875067 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 08:01:09.875466 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.875099 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs podName:c7f9811f-5a14-4945-a92a-4cb1247a34f3 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:41.875079425 +0000 UTC m=+65.266484051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs") pod "network-metrics-daemon-4tcj5" (UID: "c7f9811f-5a14-4945-a92a-4cb1247a34f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 08:01:09.875466 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.875115 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret podName:f6dd6fb1-a94d-4351-b84a-8550c44770d4 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:41.875108882 +0000 UTC m=+65.266513486 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret") pod "global-pull-secret-syncer-c2847" (UID: "f6dd6fb1-a94d-4351-b84a-8550c44770d4") : object "kube-system"/"original-pull-secret" not registered Jul 15 08:01:09.975767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:09.975737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:09.975936 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.975905 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 08:01:09.975936 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.975931 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 08:01:09.976032 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.975943 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tqc4c for pod openshift-network-diagnostics/network-check-target-nlrjg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:01:09.976032 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:09.975988 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c podName:b230ae4b-448c-4e14-ae08-d63d70ab1c0d nodeName:}" failed. No retries permitted until 2026-07-15 08:01:41.975974524 +0000 UTC m=+65.367379125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqc4c" (UniqueName: "kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c") pod "network-check-target-nlrjg" (UID: "b230ae4b-448c-4e14-ae08-d63d70ab1c0d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 08:01:10.345809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:10.345782 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:10.345958 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:10.345782 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:10.345958 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:10.345903 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:10.345958 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:10.345942 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:11.345877 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:11.345845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:11.346325 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:11.345960 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4tcj5" podUID="c7f9811f-5a14-4945-a92a-4cb1247a34f3" Jul 15 08:01:11.570594 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:11.570558 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="829093bc8cc3c4558a838e1a8a0bb491e24b2f1a53a98d5309c0e592fdcd907c" exitCode=0 Jul 15 08:01:11.570743 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:11.570612 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"829093bc8cc3c4558a838e1a8a0bb491e24b2f1a53a98d5309c0e592fdcd907c"} Jul 15 08:01:12.346309 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.346144 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:12.346670 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.346144 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:12.346670 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.346412 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nlrjg" podUID="b230ae4b-448c-4e14-ae08-d63d70ab1c0d" Jul 15 08:01:12.346670 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.346457 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2847" podUID="f6dd6fb1-a94d-4351-b84a-8550c44770d4" Jul 15 08:01:12.375320 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.375297 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-240.ec2.internal" event="NodeReady" Jul 15 08:01:12.375460 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.375443 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 08:01:12.433982 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.433953 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:01:12.478443 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.478407 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xbhjm"] Jul 15 08:01:12.478584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.478549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.489978 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.489957 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qxb7m"] Jul 15 08:01:12.490090 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.490058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.505963 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.505940 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:01:12.506053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.505969 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qxb7m"] Jul 15 08:01:12.506053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.506031 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.546642 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.546609 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 08:01:12.546642 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.546619 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 08:01:12.546889 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.546680 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 08:01:12.546889 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.546729 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 08:01:12.546889 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.546749 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 08:01:12.547401 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.547385 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xbhjm"] Jul 15 08:01:12.547803 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.547789 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 08:01:12.547967 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.547953 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 08:01:12.548020 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.547990 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 08:01:12.548257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.548240 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8tgdm\"" Jul 15 08:01:12.548386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.548371 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lkxcp\"" Jul 15 08:01:12.548438 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.548377 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2wb89\"" Jul 15 08:01:12.551583 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.551568 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 08:01:12.595917 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.595896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb6b8a5-1075-4078-a7ce-ded389856f53-tmp-dir\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.596047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.595923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt4kk\" (UniqueName: \"kubernetes.io/projected/efb6b8a5-1075-4078-a7ce-ded389856f53-kube-api-access-xt4kk\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.596047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.595939 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6733015d-b01c-482a-b99e-5891d32f5a9f-cert\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.596047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.595968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.596047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efb6b8a5-1075-4078-a7ce-ded389856f53-config-volume\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596185 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbrqz\" (UniqueName: \"kubernetes.io/projected/6733015d-b01c-482a-b99e-5891d32f5a9f-kube-api-access-dbrqz\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.596499 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.596499 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.596258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g4hg\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.696790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.696751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb6b8a5-1075-4078-a7ce-ded389856f53-tmp-dir\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.696790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.696791 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt4kk\" (UniqueName: \"kubernetes.io/projected/efb6b8a5-1075-4078-a7ce-ded389856f53-kube-api-access-xt4kk\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.697026 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.696809 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6733015d-b01c-482a-b99e-5891d32f5a9f-cert\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.697026 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.696937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.697026 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697185 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.697057 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 08:01:12.697185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697185 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.697116 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls podName:efb6b8a5-1075-4078-a7ce-ded389856f53 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:13.197094018 +0000 UTC m=+36.588498636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls") pod "dns-default-xbhjm" (UID: "efb6b8a5-1075-4078-a7ce-ded389856f53") : secret "dns-default-metrics-tls" not found Jul 15 08:01:12.697185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697149 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb6b8a5-1075-4078-a7ce-ded389856f53-tmp-dir\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efb6b8a5-1075-4078-a7ce-ded389856f53-config-volume\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697279 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbrqz\" (UniqueName: \"kubernetes.io/projected/6733015d-b01c-482a-b99e-5891d32f5a9f-kube-api-access-dbrqz\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697389 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g4hg\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.697399 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.697413 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c999f6cc-xk4w5: secret "image-registry-tls" not found Jul 15 08:01:12.697507 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:12.697456 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls podName:51b6f1fa-2e77-4d9d-a48a-1212f1f121e5 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:13.197440768 +0000 UTC m=+36.588845372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls") pod "image-registry-84c999f6cc-xk4w5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5") : secret "image-registry-tls" not found Jul 15 08:01:12.697986 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697718 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.697986 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697831 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efb6b8a5-1075-4078-a7ce-ded389856f53-config-volume\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.697986 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.697934 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.698119 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.698062 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.701203 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.701175 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.701203 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.701183 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.701203 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.701190 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6733015d-b01c-482a-b99e-5891d32f5a9f-cert\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.709373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.709338 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g4hg\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.711621 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.711601 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:12.711817 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.711798 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt4kk\" (UniqueName: \"kubernetes.io/projected/efb6b8a5-1075-4078-a7ce-ded389856f53-kube-api-access-xt4kk\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:12.712883 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.712862 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbrqz\" (UniqueName: \"kubernetes.io/projected/6733015d-b01c-482a-b99e-5891d32f5a9f-kube-api-access-dbrqz\") pod \"ingress-canary-qxb7m\" (UID: \"6733015d-b01c-482a-b99e-5891d32f5a9f\") " pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:12.814039 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:12.814008 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qxb7m" Jul 15 08:01:13.007842 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.007814 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qxb7m"] Jul 15 08:01:13.012114 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:13.012087 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6733015d_b01c_482a_b99e_5891d32f5a9f.slice/crio-9514ba577518d3d6b8e7a039ceeccb7b570c6f28656c1ac9935c7e8fff624b21 WatchSource:0}: Error finding container 9514ba577518d3d6b8e7a039ceeccb7b570c6f28656c1ac9935c7e8fff624b21: Status 404 returned error can't find the container with id 9514ba577518d3d6b8e7a039ceeccb7b570c6f28656c1ac9935c7e8fff624b21 Jul 15 08:01:13.200333 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.200251 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:13.200476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.200344 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:13.200476 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:13.200429 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 08:01:13.200476 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:13.200453 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 08:01:13.200476 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:13.200464 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c999f6cc-xk4w5: secret "image-registry-tls" not found Jul 15 08:01:13.200598 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:13.200491 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls podName:efb6b8a5-1075-4078-a7ce-ded389856f53 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:14.200476052 +0000 UTC m=+37.591880652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls") pod "dns-default-xbhjm" (UID: "efb6b8a5-1075-4078-a7ce-ded389856f53") : secret "dns-default-metrics-tls" not found Jul 15 08:01:13.200598 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:13.200505 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls podName:51b6f1fa-2e77-4d9d-a48a-1212f1f121e5 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:14.200499627 +0000 UTC m=+37.591904227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls") pod "image-registry-84c999f6cc-xk4w5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5") : secret "image-registry-tls" not found Jul 15 08:01:13.345933 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.345905 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:13.349652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.349629 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 08:01:13.350109 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.349818 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df2lv\"" Jul 15 08:01:13.576206 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:13.576123 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qxb7m" event={"ID":"6733015d-b01c-482a-b99e-5891d32f5a9f","Type":"ContainerStarted","Data":"9514ba577518d3d6b8e7a039ceeccb7b570c6f28656c1ac9935c7e8fff624b21"} Jul 15 08:01:14.208608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.208573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:14.208797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.208631 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:14.208797 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:14.208745 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 08:01:14.208797 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:14.208768 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c999f6cc-xk4w5: secret "image-registry-tls" not found Jul 15 08:01:14.208797 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:14.208780 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 08:01:14.208979 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:14.208827 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls podName:51b6f1fa-2e77-4d9d-a48a-1212f1f121e5 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:16.208808457 +0000 UTC m=+39.600213071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls") pod "image-registry-84c999f6cc-xk4w5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5") : secret "image-registry-tls" not found Jul 15 08:01:14.208979 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:14.208843 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls podName:efb6b8a5-1075-4078-a7ce-ded389856f53 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:16.208836386 +0000 UTC m=+39.600240987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls") pod "dns-default-xbhjm" (UID: "efb6b8a5-1075-4078-a7ce-ded389856f53") : secret "dns-default-metrics-tls" not found Jul 15 08:01:14.345538 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.345502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:14.345702 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.345541 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:14.351640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.351611 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 08:01:14.352031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.351667 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 08:01:14.352884 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.352703 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 08:01:14.352884 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:14.352881 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5x8qz\"" Jul 15 08:01:16.224169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.224127 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.224194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.224313 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.224338 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c999f6cc-xk4w5: secret "image-registry-tls" not found Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.224393 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.224423 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls podName:51b6f1fa-2e77-4d9d-a48a-1212f1f121e5 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:20.22440257 +0000 UTC m=+43.615807190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls") pod "image-registry-84c999f6cc-xk4w5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5") : secret "image-registry-tls" not found Jul 15 08:01:16.224635 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.224461 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls podName:efb6b8a5-1075-4078-a7ce-ded389856f53 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:20.224443733 +0000 UTC m=+43.615848339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls") pod "dns-default-xbhjm" (UID: "efb6b8a5-1075-4078-a7ce-ded389856f53") : secret "dns-default-metrics-tls" not found Jul 15 08:01:16.713125 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.713098 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pgxmz"] Jul 15 08:01:16.734554 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.734517 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgxmz"] Jul 15 08:01:16.734722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.734657 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.740834 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.740810 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 08:01:16.742277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.741946 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 08:01:16.742277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.742008 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f2tcw\"" Jul 15 08:01:16.742277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.742065 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 08:01:16.742277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.742091 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 08:01:16.828058 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.827868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359e98f9-0feb-491c-bbc2-14f0d058d07c-crio-socket\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.828227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.828083 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.828227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.828173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.828227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.828195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359e98f9-0feb-491c-bbc2-14f0d058d07c-data-volume\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.828227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.828210 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl5ww\" (UniqueName: \"kubernetes.io/projected/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-api-access-pl5ww\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929205 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929389 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359e98f9-0feb-491c-bbc2-14f0d058d07c-data-volume\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929389 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.929315 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 08:01:16.929389 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl5ww\" (UniqueName: \"kubernetes.io/projected/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-api-access-pl5ww\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929529 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.929424 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:17.42940284 +0000 UTC m=+40.820807463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : configmap "kube-rbac-proxy" not found Jul 15 08:01:16.929529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359e98f9-0feb-491c-bbc2-14f0d058d07c-crio-socket\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929735 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.929672 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:16.929735 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:16.929726 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:17.429711136 +0000 UTC m=+40.821115750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:16.929828 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929740 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359e98f9-0feb-491c-bbc2-14f0d058d07c-crio-socket\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.929871 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.929840 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359e98f9-0feb-491c-bbc2-14f0d058d07c-data-volume\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.940521 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.940496 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl5ww\" (UniqueName: \"kubernetes.io/projected/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-api-access-pl5ww\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:16.954628 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.954601 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-plj4m"] Jul 15 08:01:16.961424 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.961398 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:16.964020 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.963965 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 08:01:16.964440 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.964419 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 08:01:16.964715 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.964697 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ck757\"" Jul 15 08:01:16.969306 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:16.969246 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-plj4m"] Jul 15 08:01:17.030864 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.030833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f292e350-c244-4f24-bcf8-b25d19bbe904-nginx-conf\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.031045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.030888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.131304 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.131266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f292e350-c244-4f24-bcf8-b25d19bbe904-nginx-conf\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.131506 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.131319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.131577 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.131562 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 08:01:17.131656 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.131635 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert podName:f292e350-c244-4f24-bcf8-b25d19bbe904 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:17.631617421 +0000 UTC m=+41.023022025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-plj4m" (UID: "f292e350-c244-4f24-bcf8-b25d19bbe904") : secret "networking-console-plugin-cert" not found Jul 15 08:01:17.132091 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.132071 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f292e350-c244-4f24-bcf8-b25d19bbe904-nginx-conf\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.433548 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.433516 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:17.433906 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.433590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:17.433906 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.433680 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:17.433906 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.433734 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:18.433721304 +0000 UTC m=+41.825125908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:17.433906 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.433680 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 08:01:17.433906 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.433802 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:18.433790946 +0000 UTC m=+41.825195547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : configmap "kube-rbac-proxy" not found Jul 15 08:01:17.635620 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:17.635588 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:17.635770 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.635702 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 08:01:17.635770 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:17.635749 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert podName:f292e350-c244-4f24-bcf8-b25d19bbe904 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:18.635737693 +0000 UTC m=+42.027142294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-plj4m" (UID: "f292e350-c244-4f24-bcf8-b25d19bbe904") : secret "networking-console-plugin-cert" not found Jul 15 08:01:18.443548 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.443515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:18.444195 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.443607 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:18.444195 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.443646 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 08:01:18.444195 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.443712 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:20.44369692 +0000 UTC m=+43.835101521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : configmap "kube-rbac-proxy" not found Jul 15 08:01:18.444195 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.443735 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:18.444195 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.443782 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:20.443769635 +0000 UTC m=+43.835174241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:18.501301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.501277 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtzzx_bd80e0b2-f15c-41fa-96da-73cf4aba4e81/dns-node-resolver/0.log" Jul 15 08:01:18.590220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.590187 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qxb7m" event={"ID":"6733015d-b01c-482a-b99e-5891d32f5a9f","Type":"ContainerStarted","Data":"8f30d09785546a0f668828177fb17c6e46539b7fdb43582f6578749539deed47"} Jul 15 08:01:18.592712 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.592688 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="525547c196988c9820707a00ab19d60386f4b59c79efd69d3b71e4fc6ca1869b" exitCode=0 Jul 15 08:01:18.592813 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.592723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"525547c196988c9820707a00ab19d60386f4b59c79efd69d3b71e4fc6ca1869b"} Jul 15 08:01:18.610904 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.610854 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qxb7m" podStartSLOduration=1.9445794790000002 podStartE2EDuration="6.610836556s" totalImagesPullingTime="4.666257077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:18.610529354 +0000 UTC m=+42.001933977" watchObservedRunningTime="2026-07-15 08:01:18.610836556 +0000 UTC m=+42.002241181" Jul 15 08:01:18.645348 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.645317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:18.645512 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.645493 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 08:01:18.645584 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:18.645562 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert podName:f292e350-c244-4f24-bcf8-b25d19bbe904 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:20.645541375 +0000 UTC m=+44.036945978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-plj4m" (UID: "f292e350-c244-4f24-bcf8-b25d19bbe904") : secret "networking-console-plugin-cert" not found Jul 15 08:01:18.682210 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.682184 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l8j46"] Jul 15 08:01:18.715909 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.715856 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l8j46"] Jul 15 08:01:18.715993 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.715947 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.721530 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.721436 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 08:01:18.721994 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.721914 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 08:01:18.722097 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.722011 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hr5vp\"" Jul 15 08:01:18.722588 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.722563 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 08:01:18.722650 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.722604 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 08:01:18.846549 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.846523 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-key\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.846675 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.846555 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvqfh\" (UniqueName: \"kubernetes.io/projected/147fed3d-bfaf-4336-aaa7-31a8236001a0-kube-api-access-kvqfh\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.846675 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.846595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-cabundle\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.947479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.947458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-key\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.947600 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.947483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqfh\" (UniqueName: \"kubernetes.io/projected/147fed3d-bfaf-4336-aaa7-31a8236001a0-kube-api-access-kvqfh\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.947600 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.947514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-cabundle\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.948810 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.948606 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-cabundle\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.952282 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.952259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/147fed3d-bfaf-4336-aaa7-31a8236001a0-signing-key\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:18.957620 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:18.957595 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvqfh\" (UniqueName: \"kubernetes.io/projected/147fed3d-bfaf-4336-aaa7-31a8236001a0-kube-api-access-kvqfh\") pod \"service-ca-79f68b9f4f-l8j46\" (UID: \"147fed3d-bfaf-4336-aaa7-31a8236001a0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:19.023938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.023877 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" Jul 15 08:01:19.174436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.174409 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-l8j46"] Jul 15 08:01:19.175185 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:19.175158 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147fed3d_bfaf_4336_aaa7_31a8236001a0.slice/crio-345b8d773a2a4f7b5c71b3f3d5562d47109b6b9aaf7ec76172ebc16101cf4376 WatchSource:0}: Error finding container 345b8d773a2a4f7b5c71b3f3d5562d47109b6b9aaf7ec76172ebc16101cf4376: Status 404 returned error can't find the container with id 345b8d773a2a4f7b5c71b3f3d5562d47109b6b9aaf7ec76172ebc16101cf4376 Jul 15 08:01:19.512347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.512312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zcxrr_7ddf0841-975e-4676-a0c1-a7133fa9549d/node-ca/0.log" Jul 15 08:01:19.595775 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.595736 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" event={"ID":"147fed3d-bfaf-4336-aaa7-31a8236001a0","Type":"ContainerStarted","Data":"345b8d773a2a4f7b5c71b3f3d5562d47109b6b9aaf7ec76172ebc16101cf4376"} Jul 15 08:01:19.598463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.598430 2581 generic.go:356] "Generic (PLEG): container finished" podID="8425c324-9b90-4ddb-9025-00fb87f97531" containerID="5e208cc8ffa41397624cf2b76e4cb5d352ea3629537d3f60ef621e63706aeb16" exitCode=0 Jul 15 08:01:19.598578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:19.598475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerDied","Data":"5e208cc8ffa41397624cf2b76e4cb5d352ea3629537d3f60ef621e63706aeb16"} Jul 15 08:01:20.102520 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.102491 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qxb7m_6733015d-b01c-482a-b99e-5891d32f5a9f/serve-healthcheck-canary/0.log" Jul 15 08:01:20.259904 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.259857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:20.260073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.259971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:20.260073 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.260018 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 08:01:20.260073 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.260057 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 08:01:20.260073 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.260073 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c999f6cc-xk4w5: secret "image-registry-tls" not found Jul 15 08:01:20.260205 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.260092 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls podName:efb6b8a5-1075-4078-a7ce-ded389856f53 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:28.260072426 +0000 UTC m=+51.651477045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls") pod "dns-default-xbhjm" (UID: "efb6b8a5-1075-4078-a7ce-ded389856f53") : secret "dns-default-metrics-tls" not found Jul 15 08:01:20.260205 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.260113 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls podName:51b6f1fa-2e77-4d9d-a48a-1212f1f121e5 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:28.260102684 +0000 UTC m=+51.651507284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls") pod "image-registry-84c999f6cc-xk4w5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5") : secret "image-registry-tls" not found Jul 15 08:01:20.460893 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.460808 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:20.461030 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.460918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:20.461030 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.460958 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 08:01:20.461030 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.461023 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:20.461141 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.461036 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:24.461015533 +0000 UTC m=+47.852420164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : configmap "kube-rbac-proxy" not found Jul 15 08:01:20.461141 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.461061 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:24.461051054 +0000 UTC m=+47.852455658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:20.604747 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.604705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pm62g" event={"ID":"8425c324-9b90-4ddb-9025-00fb87f97531","Type":"ContainerStarted","Data":"d676fed107c588324979506dfa27399a064ffe2496dda513da254b51cc9f9ec0"} Jul 15 08:01:20.639199 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.639154 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pm62g" podStartSLOduration=8.159468082 podStartE2EDuration="43.639139139s" totalImagesPullingTime="35.479671057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:20.637844973 +0000 UTC m=+44.029249596" watchObservedRunningTime="2026-07-15 08:01:20.639139139 +0000 UTC m=+44.030543761" Jul 15 08:01:20.662495 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:20.662470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:20.662617 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.662601 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 08:01:20.662673 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:20.662668 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert podName:f292e350-c244-4f24-bcf8-b25d19bbe904 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:24.662654789 +0000 UTC m=+48.054059390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-plj4m" (UID: "f292e350-c244-4f24-bcf8-b25d19bbe904") : secret "networking-console-plugin-cert" not found Jul 15 08:01:22.610796 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:22.610763 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" event={"ID":"147fed3d-bfaf-4336-aaa7-31a8236001a0","Type":"ContainerStarted","Data":"c222271baff8638fe3ed51d5965c2d7a0bb3680f023832ecd08b5cee599605ef"} Jul 15 08:01:22.628806 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:22.628755 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-l8j46" podStartSLOduration=2.117855297 podStartE2EDuration="4.628740695s" totalImagesPullingTime="2.510885398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:22.628371079 +0000 UTC m=+46.019775695" watchObservedRunningTime="2026-07-15 08:01:22.628740695 +0000 UTC m=+46.020145318" Jul 15 08:01:24.493840 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:24.493798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:24.494231 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:24.493890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:24.494231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.493929 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 08:01:24.494231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.493992 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:24.494231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.493993 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:32.493977776 +0000 UTC m=+55.885382381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : configmap "kube-rbac-proxy" not found Jul 15 08:01:24.494231 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.494039 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:32.494027717 +0000 UTC m=+55.885432318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:24.695408 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:24.695351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:24.695567 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.695517 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 08:01:24.695608 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:24.695596 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert podName:f292e350-c244-4f24-bcf8-b25d19bbe904 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:32.695581043 +0000 UTC m=+56.086985643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-plj4m" (UID: "f292e350-c244-4f24-bcf8-b25d19bbe904") : secret "networking-console-plugin-cert" not found Jul 15 08:01:28.327731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.327696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:28.328099 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.327753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:28.332717 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.332695 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"image-registry-84c999f6cc-xk4w5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:28.340927 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.340906 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efb6b8a5-1075-4078-a7ce-ded389856f53-metrics-tls\") pod \"dns-default-xbhjm\" (UID: \"efb6b8a5-1075-4078-a7ce-ded389856f53\") " pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:28.387506 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.387471 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:28.397589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.397563 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:28.543404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.543344 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:01:28.546157 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:28.546130 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b6f1fa_2e77_4d9d_a48a_1212f1f121e5.slice/crio-07727bc480aa2047237bff078e06b2af0b2632ecb859dbfb9a455aaf3bcc4fb0 WatchSource:0}: Error finding container 07727bc480aa2047237bff078e06b2af0b2632ecb859dbfb9a455aaf3bcc4fb0: Status 404 returned error can't find the container with id 07727bc480aa2047237bff078e06b2af0b2632ecb859dbfb9a455aaf3bcc4fb0 Jul 15 08:01:28.558820 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.558794 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xbhjm"] Jul 15 08:01:28.561282 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:28.561260 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb6b8a5_1075_4078_a7ce_ded389856f53.slice/crio-4dcd6af60057c94e914430054c848665db65feafe8959bc84b102c5256bc4d40 WatchSource:0}: Error finding container 4dcd6af60057c94e914430054c848665db65feafe8959bc84b102c5256bc4d40: Status 404 returned error can't find the container with id 4dcd6af60057c94e914430054c848665db65feafe8959bc84b102c5256bc4d40 Jul 15 08:01:28.626764 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.626733 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" event={"ID":"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5","Type":"ContainerStarted","Data":"07727bc480aa2047237bff078e06b2af0b2632ecb859dbfb9a455aaf3bcc4fb0"} Jul 15 08:01:28.627706 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:28.627688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xbhjm" event={"ID":"efb6b8a5-1075-4078-a7ce-ded389856f53","Type":"ContainerStarted","Data":"4dcd6af60057c94e914430054c848665db65feafe8959bc84b102c5256bc4d40"} Jul 15 08:01:29.632397 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:29.632303 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" event={"ID":"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5","Type":"ContainerStarted","Data":"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863"} Jul 15 08:01:29.632855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:29.632548 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:31.637643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:31.637616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xbhjm" event={"ID":"efb6b8a5-1075-4078-a7ce-ded389856f53","Type":"ContainerStarted","Data":"d550044a7cff581a8c5fd13f6d1d3c77d39542e5fda2f0b64f30aae615573891"} Jul 15 08:01:32.567255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.567216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:32.567480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.567313 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:32.567480 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:32.567450 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 08:01:32.567600 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:32.567520 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls podName:359e98f9-0feb-491c-bbc2-14f0d058d07c nodeName:}" failed. No retries permitted until 2026-07-15 08:01:48.567501051 +0000 UTC m=+71.958905673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgxmz" (UID: "359e98f9-0feb-491c-bbc2-14f0d058d07c") : secret "insights-runtime-extractor-tls" not found Jul 15 08:01:32.567882 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.567859 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359e98f9-0feb-491c-bbc2-14f0d058d07c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:32.643341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.643305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xbhjm" event={"ID":"efb6b8a5-1075-4078-a7ce-ded389856f53","Type":"ContainerStarted","Data":"866fc392ac43adaf446fae5f13bee810cac709d9830120a92880752128939440"} Jul 15 08:01:32.643808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.643423 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:32.662093 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.662038 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" podStartSLOduration=22.662019803 podStartE2EDuration="22.662019803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:29.654173484 +0000 UTC m=+53.045578108" watchObservedRunningTime="2026-07-15 08:01:32.662019803 +0000 UTC m=+56.053424427" Jul 15 08:01:32.662726 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.662694 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xbhjm" podStartSLOduration=17.786783301 podStartE2EDuration="20.662683806s" totalImagesPullingTime="2.875900505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:32.661178729 +0000 UTC m=+56.052583373" watchObservedRunningTime="2026-07-15 08:01:32.662683806 +0000 UTC m=+56.054088428" Jul 15 08:01:32.769087 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.769048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:32.771445 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.771427 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f292e350-c244-4f24-bcf8-b25d19bbe904-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-plj4m\" (UID: \"f292e350-c244-4f24-bcf8-b25d19bbe904\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:32.872743 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.872713 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" Jul 15 08:01:32.997125 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:32.995319 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-plj4m"] Jul 15 08:01:32.999710 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:32.999682 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf292e350_c244_4f24_bcf8_b25d19bbe904.slice/crio-6df5d32dd55e46ba9d8b0ee948a03ed0bc53d2eedf8d70f1bb75525588edcc91 WatchSource:0}: Error finding container 6df5d32dd55e46ba9d8b0ee948a03ed0bc53d2eedf8d70f1bb75525588edcc91: Status 404 returned error can't find the container with id 6df5d32dd55e46ba9d8b0ee948a03ed0bc53d2eedf8d70f1bb75525588edcc91 Jul 15 08:01:33.647190 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:33.647151 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" event={"ID":"f292e350-c244-4f24-bcf8-b25d19bbe904","Type":"ContainerStarted","Data":"6df5d32dd55e46ba9d8b0ee948a03ed0bc53d2eedf8d70f1bb75525588edcc91"} Jul 15 08:01:34.651645 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:34.651611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" event={"ID":"f292e350-c244-4f24-bcf8-b25d19bbe904","Type":"ContainerStarted","Data":"b3131575c44bd6bf619cad3ab4aa65a2b49fd396d04efc6e56aa9f220e0b022e"} Jul 15 08:01:34.668955 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:34.668905 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-plj4m" podStartSLOduration=17.428785723 podStartE2EDuration="18.668889421s" totalImagesPullingTime="1.240103698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:34.667731843 +0000 UTC m=+58.059136485" watchObservedRunningTime="2026-07-15 08:01:34.668889421 +0000 UTC m=+58.060294044" Jul 15 08:01:38.571549 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:38.571519 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-htgqh" Jul 15 08:01:40.290814 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.290774 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2"] Jul 15 08:01:40.294062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.294021 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.296698 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.296678 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 15 08:01:40.296938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.296918 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 08:01:40.297013 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.296923 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 08:01:40.297701 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.297682 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 15 08:01:40.297812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.297689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 15 08:01:40.297812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.297728 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 15 08:01:40.297812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.297771 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 08:01:40.302750 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.302726 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2"] Jul 15 08:01:40.327055 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.327245 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.327245 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.327245 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxs2s\" (UniqueName: \"kubernetes.io/projected/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-kube-api-access-lxs2s\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.327444 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.327444 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.327403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.395181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.395145 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:01:40.428055 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.428222 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.428293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.428338 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxs2s\" (UniqueName: \"kubernetes.io/projected/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-kube-api-access-lxs2s\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.428483 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.428547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.428498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.429257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.429230 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.430690 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.430663 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.430760 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.430701 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-hub\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.430760 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.430732 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.430839 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.430733 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-ca\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.440261 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.440237 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxs2s\" (UniqueName: \"kubernetes.io/projected/281229fe-194f-44ad-ac7e-0b5fa7b7aaac-kube-api-access-lxs2s\") pod \"cluster-proxy-proxy-agent-5f947548b7-kfjr2\" (UID: \"281229fe-194f-44ad-ac7e-0b5fa7b7aaac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.614034 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.613984 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" Jul 15 08:01:40.750601 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:40.750567 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2"] Jul 15 08:01:40.753596 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:40.753568 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod281229fe_194f_44ad_ac7e_0b5fa7b7aaac.slice/crio-2bff2e878afda043498cccf4b57e5175c0d9ca917509935bf09bf8b2a0022327 WatchSource:0}: Error finding container 2bff2e878afda043498cccf4b57e5175c0d9ca917509935bf09bf8b2a0022327: Status 404 returned error can't find the container with id 2bff2e878afda043498cccf4b57e5175c0d9ca917509935bf09bf8b2a0022327 Jul 15 08:01:41.676933 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.676887 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" event={"ID":"281229fe-194f-44ad-ac7e-0b5fa7b7aaac","Type":"ContainerStarted","Data":"2bff2e878afda043498cccf4b57e5175c0d9ca917509935bf09bf8b2a0022327"} Jul 15 08:01:41.941761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.941667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:41.941761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.941709 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:41.944375 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.944330 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 08:01:41.944531 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.944415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 08:01:41.955391 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.955344 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f6dd6fb1-a94d-4351-b84a-8550c44770d4-original-pull-secret\") pod \"global-pull-secret-syncer-c2847\" (UID: \"f6dd6fb1-a94d-4351-b84a-8550c44770d4\") " pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:41.955391 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.955376 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9811f-5a14-4945-a92a-4cb1247a34f3-metrics-certs\") pod \"network-metrics-daemon-4tcj5\" (UID: \"c7f9811f-5a14-4945-a92a-4cb1247a34f3\") " pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:41.960301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:41.960268 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2847" Jul 15 08:01:42.042928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.042683 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:42.045562 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.045536 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 08:01:42.055971 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.055886 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 08:01:42.068014 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.067945 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqc4c\" (UniqueName: \"kubernetes.io/projected/b230ae4b-448c-4e14-ae08-d63d70ab1c0d-kube-api-access-tqc4c\") pod \"network-check-target-nlrjg\" (UID: \"b230ae4b-448c-4e14-ae08-d63d70ab1c0d\") " pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:42.119478 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.119439 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c2847"] Jul 15 08:01:42.123116 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:42.123091 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dd6fb1_a94d_4351_b84a_8550c44770d4.slice/crio-89bffe7b3f3f28bf30428b8c58f4588b79f0c5b212eb17b97ef9c49dbd9e42f8 WatchSource:0}: Error finding container 89bffe7b3f3f28bf30428b8c58f4588b79f0c5b212eb17b97ef9c49dbd9e42f8: Status 404 returned error can't find the container with id 89bffe7b3f3f28bf30428b8c58f4588b79f0c5b212eb17b97ef9c49dbd9e42f8 Jul 15 08:01:42.159264 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.159224 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df2lv\"" Jul 15 08:01:42.167415 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.167337 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4tcj5" Jul 15 08:01:42.269786 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.269701 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5x8qz\"" Jul 15 08:01:42.277541 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.277510 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:42.649960 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.649926 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xbhjm" Jul 15 08:01:42.682150 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:42.682069 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c2847" event={"ID":"f6dd6fb1-a94d-4351-b84a-8550c44770d4","Type":"ContainerStarted","Data":"89bffe7b3f3f28bf30428b8c58f4588b79f0c5b212eb17b97ef9c49dbd9e42f8"} Jul 15 08:01:43.080876 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:43.080842 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4tcj5"] Jul 15 08:01:43.099158 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:43.099126 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nlrjg"] Jul 15 08:01:43.102929 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:43.102899 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb230ae4b_448c_4e14_ae08_d63d70ab1c0d.slice/crio-eb5f1efc791a3d19637bf0fc17f1ce9b697b2753f87eadd3ad946e32155aa023 WatchSource:0}: Error finding container eb5f1efc791a3d19637bf0fc17f1ce9b697b2753f87eadd3ad946e32155aa023: Status 404 returned error can't find the container with id eb5f1efc791a3d19637bf0fc17f1ce9b697b2753f87eadd3ad946e32155aa023 Jul 15 08:01:43.686999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:43.686959 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nlrjg" event={"ID":"b230ae4b-448c-4e14-ae08-d63d70ab1c0d","Type":"ContainerStarted","Data":"eb5f1efc791a3d19637bf0fc17f1ce9b697b2753f87eadd3ad946e32155aa023"} Jul 15 08:01:43.688432 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:43.688392 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tcj5" event={"ID":"c7f9811f-5a14-4945-a92a-4cb1247a34f3","Type":"ContainerStarted","Data":"a24de5d056e1192bbae531bf9d42718e394fdc14fff4cd4fb4aae6f670c3d4bb"} Jul 15 08:01:44.695238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:44.695159 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" event={"ID":"281229fe-194f-44ad-ac7e-0b5fa7b7aaac","Type":"ContainerStarted","Data":"272a387d73e7bc2a4cc74518e145a6b22f372e37f87bbe9476ebfac601231d30"} Jul 15 08:01:44.695238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:44.695208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" event={"ID":"281229fe-194f-44ad-ac7e-0b5fa7b7aaac","Type":"ContainerStarted","Data":"c0373a90ad201faf596ad9ec63dfe3daaa61ed3e9e3a0b90d706d374dd6a42ce"} Jul 15 08:01:45.703095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:45.702568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" event={"ID":"281229fe-194f-44ad-ac7e-0b5fa7b7aaac","Type":"ContainerStarted","Data":"ef24f5cc29f4a2d9c1768db760fff4572b32ae49f247830a5bff7a7499fd6f8c"} Jul 15 08:01:45.704676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:45.704649 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tcj5" event={"ID":"c7f9811f-5a14-4945-a92a-4cb1247a34f3","Type":"ContainerStarted","Data":"66aba8de82ffae3ddc30f5788ee3a630ee901ea90d063373d4c9cea7d91f60c7"} Jul 15 08:01:45.704817 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:45.704714 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4tcj5" event={"ID":"c7f9811f-5a14-4945-a92a-4cb1247a34f3","Type":"ContainerStarted","Data":"1b1d78585aa828664e9ad961170a97bf4ee35f75274f8c35c03de34a86efbccb"} Jul 15 08:01:45.729301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:45.729231 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f947548b7-kfjr2" podStartSLOduration=1.994239511 podStartE2EDuration="5.729210907s" totalImagesPullingTime="3.734971396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:45.72833459 +0000 UTC m=+69.119739225" watchObservedRunningTime="2026-07-15 08:01:45.729210907 +0000 UTC m=+69.120615530" Jul 15 08:01:45.749976 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:45.749888 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4tcj5" podStartSLOduration=66.646851659 podStartE2EDuration="1m8.74982684s" totalImagesPullingTime="2.102975181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:45.748744646 +0000 UTC m=+69.140149294" watchObservedRunningTime="2026-07-15 08:01:45.74982684 +0000 UTC m=+69.141231464" Jul 15 08:01:48.602663 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.602607 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:48.604935 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.604909 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359e98f9-0feb-491c-bbc2-14f0d058d07c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgxmz\" (UID: \"359e98f9-0feb-491c-bbc2-14f0d058d07c\") " pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:48.717208 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.717165 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nlrjg" event={"ID":"b230ae4b-448c-4e14-ae08-d63d70ab1c0d","Type":"ContainerStarted","Data":"31b5a80ce37efdc4ddbebc2fa75664d9b99f39890cb3f643a78095aed4a3ac50"} Jul 15 08:01:48.717440 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.717265 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:01:48.718642 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.718611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c2847" event={"ID":"f6dd6fb1-a94d-4351-b84a-8550c44770d4","Type":"ContainerStarted","Data":"f99c36541ec3ba7e36955d55f3540b2728fafd0ed9667a66a479ab49f929f4dc"} Jul 15 08:01:48.736528 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.736481 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nlrjg" podStartSLOduration=66.789331172 podStartE2EDuration="1m11.736466831s" totalImagesPullingTime="4.947135659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:48.735792748 +0000 UTC m=+72.127197389" watchObservedRunningTime="2026-07-15 08:01:48.736466831 +0000 UTC m=+72.127871453" Jul 15 08:01:48.751973 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.751926 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c2847" podStartSLOduration=65.824075141 podStartE2EDuration="1m11.751910587s" totalImagesPullingTime="5.927835446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:48.751043184 +0000 UTC m=+72.142447808" watchObservedRunningTime="2026-07-15 08:01:48.751910587 +0000 UTC m=+72.143315210" Jul 15 08:01:48.847320 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.847286 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f2tcw\"" Jul 15 08:01:48.855511 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.855449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgxmz" Jul 15 08:01:48.982635 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:48.982599 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgxmz"] Jul 15 08:01:48.985820 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:48.985799 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359e98f9_0feb_491c_bbc2_14f0d058d07c.slice/crio-45459b1d45eadade94167934b150cb251b7c480c1d030cd1617c5ed203178607 WatchSource:0}: Error finding container 45459b1d45eadade94167934b150cb251b7c480c1d030cd1617c5ed203178607: Status 404 returned error can't find the container with id 45459b1d45eadade94167934b150cb251b7c480c1d030cd1617c5ed203178607 Jul 15 08:01:49.724778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:49.724742 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgxmz" event={"ID":"359e98f9-0feb-491c-bbc2-14f0d058d07c","Type":"ContainerStarted","Data":"45459b1d45eadade94167934b150cb251b7c480c1d030cd1617c5ed203178607"} Jul 15 08:01:49.997626 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:49.997519 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cq42h"] Jul 15 08:01:50.008727 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.008700 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.012215 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.012186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 08:01:50.012373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.012332 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 08:01:50.012373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.012331 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 08:01:50.013436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.013332 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 08:01:50.013436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.013422 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4qvg4\"" Jul 15 08:01:50.013727 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.013565 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 08:01:50.017747 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.017724 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cq42h"] Jul 15 08:01:50.115486 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.115457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4knqb\" (UniqueName: \"kubernetes.io/projected/183a9041-0728-4f42-b9ce-a608eed01d56-kube-api-access-4knqb\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.115652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.115505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.115652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.115530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183a9041-0728-4f42-b9ce-a608eed01d56-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.115652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.115627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.216049 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.216017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.216049 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.216052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183a9041-0728-4f42-b9ce-a608eed01d56-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.216236 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:50.216164 2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 15 08:01:50.216236 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.216185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.216236 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:01:50.216222 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls podName:183a9041-0728-4f42-b9ce-a608eed01d56 nodeName:}" failed. No retries permitted until 2026-07-15 08:01:50.716207807 +0000 UTC m=+74.107612408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls") pod "prometheus-operator-64785b6675-cq42h" (UID: "183a9041-0728-4f42-b9ce-a608eed01d56") : secret "prometheus-operator-tls" not found Jul 15 08:01:50.216396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.216276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4knqb\" (UniqueName: \"kubernetes.io/projected/183a9041-0728-4f42-b9ce-a608eed01d56-kube-api-access-4knqb\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.216742 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.216721 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183a9041-0728-4f42-b9ce-a608eed01d56-metrics-client-ca\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.218570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.218547 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.225938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.225914 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4knqb\" (UniqueName: \"kubernetes.io/projected/183a9041-0728-4f42-b9ce-a608eed01d56-kube-api-access-4knqb\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.400434 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.400410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:01:50.721759 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.721675 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.724570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.724545 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183a9041-0728-4f42-b9ce-a608eed01d56-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-cq42h\" (UID: \"183a9041-0728-4f42-b9ce-a608eed01d56\") " pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:50.921374 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:50.921324 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" Jul 15 08:01:51.102249 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:51.102220 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-cq42h"] Jul 15 08:01:51.105106 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:51.105072 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183a9041_0728_4f42_b9ce_a608eed01d56.slice/crio-4a41b418df2b1d45272092b9bcc23a10af32d5352ee918e489baa78447689155 WatchSource:0}: Error finding container 4a41b418df2b1d45272092b9bcc23a10af32d5352ee918e489baa78447689155: Status 404 returned error can't find the container with id 4a41b418df2b1d45272092b9bcc23a10af32d5352ee918e489baa78447689155 Jul 15 08:01:51.737164 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:51.737130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" event={"ID":"183a9041-0728-4f42-b9ce-a608eed01d56","Type":"ContainerStarted","Data":"4a41b418df2b1d45272092b9bcc23a10af32d5352ee918e489baa78447689155"} Jul 15 08:01:54.750976 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:54.750940 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgxmz" event={"ID":"359e98f9-0feb-491c-bbc2-14f0d058d07c","Type":"ContainerStarted","Data":"8950c6a805be54b54e19855441cda7f34ff9f6f7b024bfef7c19fcff9e02b805"} Jul 15 08:01:55.756681 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:55.756643 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" event={"ID":"183a9041-0728-4f42-b9ce-a608eed01d56","Type":"ContainerStarted","Data":"fd6372d398bd210aeebbc58eda63c85e769a571afae0f871a0b4e2bc0ba8b66a"} Jul 15 08:01:55.757102 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:55.756700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" event={"ID":"183a9041-0728-4f42-b9ce-a608eed01d56","Type":"ContainerStarted","Data":"77a2475b5b6121dbaa0f49b468795b1e4a3b4a17dddbcb9929e284af7dc2eff1"} Jul 15 08:01:55.777894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:55.777840 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-cq42h" podStartSLOduration=2.959514479 podStartE2EDuration="6.777824411s" totalImagesPullingTime="3.818309932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:01:55.776081562 +0000 UTC m=+79.167486217" watchObservedRunningTime="2026-07-15 08:01:55.777824411 +0000 UTC m=+79.169229033" Jul 15 08:01:57.441048 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.441011 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9m5jt"] Jul 15 08:01:57.444943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.444913 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.448452 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.448429 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 08:01:57.448575 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.448514 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fqpfw\"" Jul 15 08:01:57.448809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.448785 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 08:01:57.448899 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.448785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 08:01:57.585188 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-wtmp\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585199 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-metrics-client-ca\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585235 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-sys\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585294 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-textfile\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-root\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585691 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-tls\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.585691 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.585422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9wth\" (UniqueName: \"kubernetes.io/projected/9e607e94-651c-479b-80e1-ee44caa4177f-kube-api-access-z9wth\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686816 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686776 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-textfile\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-root\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-tls\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9wth\" (UniqueName: \"kubernetes.io/projected/9e607e94-651c-479b-80e1-ee44caa4177f-kube-api-access-z9wth\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-wtmp\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-metrics-client-ca\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.686990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.686986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-sys\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687057 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-textfile\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687228 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-wtmp\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687263 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-root\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687596 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e607e94-651c-479b-80e1-ee44caa4177f-sys\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.687894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.687646 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.688139 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.688028 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e607e94-651c-479b-80e1-ee44caa4177f-metrics-client-ca\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.690076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.690033 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-tls\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.690493 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.690466 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e607e94-651c-479b-80e1-ee44caa4177f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.696566 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.696509 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9wth\" (UniqueName: \"kubernetes.io/projected/9e607e94-651c-479b-80e1-ee44caa4177f-kube-api-access-z9wth\") pod \"node-exporter-9m5jt\" (UID: \"9e607e94-651c-479b-80e1-ee44caa4177f\") " pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:57.757950 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:57.757922 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9m5jt" Jul 15 08:01:58.064101 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:01:58.064064 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e607e94_651c_479b_80e1_ee44caa4177f.slice/crio-05dc695f6db6028f409e2508f2577313a34067c17088f1d0eb85b9667dd34557 WatchSource:0}: Error finding container 05dc695f6db6028f409e2508f2577313a34067c17088f1d0eb85b9667dd34557: Status 404 returned error can't find the container with id 05dc695f6db6028f409e2508f2577313a34067c17088f1d0eb85b9667dd34557 Jul 15 08:01:58.769581 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:58.769543 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9m5jt" event={"ID":"9e607e94-651c-479b-80e1-ee44caa4177f","Type":"ContainerStarted","Data":"05dc695f6db6028f409e2508f2577313a34067c17088f1d0eb85b9667dd34557"} Jul 15 08:01:58.771671 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:58.771646 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgxmz" event={"ID":"359e98f9-0feb-491c-bbc2-14f0d058d07c","Type":"ContainerStarted","Data":"0f8f2c78b1e2e96594834c1754bac146d5eaf6098828a9696be8b073a17aed30"} Jul 15 08:01:59.776814 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:59.776703 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e607e94-651c-479b-80e1-ee44caa4177f" containerID="78bb15f8833acdb0c6b9365e50083be716b6add30f5edb2fb6a58a3d9a8b8754" exitCode=0 Jul 15 08:01:59.776814 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:01:59.776794 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9m5jt" event={"ID":"9e607e94-651c-479b-80e1-ee44caa4177f","Type":"ContainerDied","Data":"78bb15f8833acdb0c6b9365e50083be716b6add30f5edb2fb6a58a3d9a8b8754"} Jul 15 08:02:00.783837 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:00.783791 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9m5jt" event={"ID":"9e607e94-651c-479b-80e1-ee44caa4177f","Type":"ContainerStarted","Data":"9ca05b31c8f4153d079d884f62e4944e13178a728bb8792bf72589bcc58c327c"} Jul 15 08:02:00.784291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:00.783890 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9m5jt" event={"ID":"9e607e94-651c-479b-80e1-ee44caa4177f","Type":"ContainerStarted","Data":"07a3a6f061610e7c0f6520a9c2b94f3248d0c0557213354cd9cbc236e51f8865"} Jul 15 08:02:00.809131 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:00.809087 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9m5jt" podStartSLOduration=2.865650519 podStartE2EDuration="3.809072855s" totalImagesPullingTime="880.224782ms" totalInitContainerRuntime="63.197554ms" isStatefulPod=true podCreationTimestamp="2026-07-15 08:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:02:00.807683573 +0000 UTC m=+84.199088225" watchObservedRunningTime="2026-07-15 08:02:00.809072855 +0000 UTC m=+84.200477477" Jul 15 08:02:03.797207 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:03.797170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgxmz" event={"ID":"359e98f9-0feb-491c-bbc2-14f0d058d07c","Type":"ContainerStarted","Data":"121d71b889fc5bc48881c062f3aadb85db955b8ee4b67ee801b278b1ac7f25a5"} Jul 15 08:02:03.821839 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:03.821777 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pgxmz" podStartSLOduration=33.741583308 podStartE2EDuration="47.821755203s" totalImagesPullingTime="14.080171895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:01:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:02:03.821242544 +0000 UTC m=+87.212647188" watchObservedRunningTime="2026-07-15 08:02:03.821755203 +0000 UTC m=+87.213159827" Jul 15 08:02:04.339045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.339005 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:04.345801 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.345770 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.348671 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.348648 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 08:02:04.348799 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.348734 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 08:02:04.349667 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349644 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 08:02:04.349776 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349646 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 08:02:04.349776 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349646 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 08:02:04.349776 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349737 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kcpks\"" Jul 15 08:02:04.349920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349897 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 08:02:04.350028 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.349982 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 08:02:04.355988 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.355954 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:04.359149 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.359121 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 08:02:04.444291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444251 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjc4c\" (UniqueName: \"kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444635 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444494 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.444635 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.444548 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545417 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjc4c\" (UniqueName: \"kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545584 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545763 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.545763 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.545632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.546197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.546168 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.546341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.546241 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.546419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.546271 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.546464 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.546443 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.547865 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.547844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.548058 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.548038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.554660 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.554642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjc4c\" (UniqueName: \"kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c\") pod \"console-7648c846ff-srzln\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.660496 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.660395 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:04.792370 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.792192 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:04.794847 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:02:04.794823 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17331907_69af_4fcd_920b_547d27146b46.slice/crio-883fe0e0606ad8bb14f430927b44288356af795ce33ea078813977d73bdc6299 WatchSource:0}: Error finding container 883fe0e0606ad8bb14f430927b44288356af795ce33ea078813977d73bdc6299: Status 404 returned error can't find the container with id 883fe0e0606ad8bb14f430927b44288356af795ce33ea078813977d73bdc6299 Jul 15 08:02:04.802191 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:04.802167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648c846ff-srzln" event={"ID":"17331907-69af-4fcd-920b-547d27146b46","Type":"ContainerStarted","Data":"883fe0e0606ad8bb14f430927b44288356af795ce33ea078813977d73bdc6299"} Jul 15 08:02:05.414802 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.414764 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" podUID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" containerName="registry" containerID="cri-o://feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863" gracePeriod=30 Jul 15 08:02:05.673285 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.673222 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:02:05.753646 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753587 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.753832 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753767 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.753832 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753803 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.753922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753858 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.753922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753892 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.754019 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753923 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.754019 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753967 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.754117 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.753996 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g4hg\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg\") pod \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\" (UID: \"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5\") " Jul 15 08:02:05.754926 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.754754 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:05.755064 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.755044 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:05.756804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.756766 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:02:05.756946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.756858 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:02:05.756946 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.756912 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:02:05.757613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.757583 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:02:05.757717 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.757591 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg" (OuterVolumeSpecName: "kube-api-access-8g4hg") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "kube-api-access-8g4hg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:02:05.765722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.765684 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" (UID: "51b6f1fa-2e77-4d9d-a48a-1212f1f121e5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:02:05.807750 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.807713 2581 generic.go:356] "Generic (PLEG): container finished" podID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" containerID="feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863" exitCode=0 Jul 15 08:02:05.808153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.807771 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" Jul 15 08:02:05.808153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.807806 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" event={"ID":"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5","Type":"ContainerDied","Data":"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863"} Jul 15 08:02:05.808153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.807869 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84c999f6cc-xk4w5" event={"ID":"51b6f1fa-2e77-4d9d-a48a-1212f1f121e5","Type":"ContainerDied","Data":"07727bc480aa2047237bff078e06b2af0b2632ecb859dbfb9a455aaf3bcc4fb0"} Jul 15 08:02:05.808153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.807895 2581 scope.go:117] "RemoveContainer" containerID="feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863" Jul 15 08:02:05.816768 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.816753 2581 scope.go:117] "RemoveContainer" containerID="feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863" Jul 15 08:02:05.817057 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:02:05.817035 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863\": container with ID starting with feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863 not found: ID does not exist" containerID="feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863" Jul 15 08:02:05.817126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.817066 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863"} err="failed to get container status \"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863\": rpc error: code = NotFound desc = could not find container \"feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863\": container with ID starting with feb2a42f4467e6b6f07300babb5f4aacac9bd50ff9747c8f51a9f1cec1c12863 not found: ID does not exist" Jul 15 08:02:05.832711 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.832684 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:02:05.835174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.835151 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84c999f6cc-xk4w5"] Jul 15 08:02:05.854771 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854742 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-tls\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854771 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854767 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-installation-pull-secrets\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854771 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854776 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-image-registry-private-configuration\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854787 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-trusted-ca\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854796 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-ca-trust-extracted\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854804 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-registry-certificates\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854816 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g4hg\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-kube-api-access-8g4hg\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:05.854959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:05.854827 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5-bound-sa-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:07.351087 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:07.351041 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" path="/var/lib/kubelet/pods/51b6f1fa-2e77-4d9d-a48a-1212f1f121e5/volumes" Jul 15 08:02:07.824141 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:07.824111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648c846ff-srzln" event={"ID":"17331907-69af-4fcd-920b-547d27146b46","Type":"ContainerStarted","Data":"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c"} Jul 15 08:02:07.849998 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:07.849929 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7648c846ff-srzln" podStartSLOduration=0.903231383 podStartE2EDuration="3.849910311s" totalImagesPullingTime="2.946678928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:02:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:02:07.848658841 +0000 UTC m=+91.240063464" watchObservedRunningTime="2026-07-15 08:02:07.849910311 +0000 UTC m=+91.241314936" Jul 15 08:02:08.435077 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.435045 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:08.471524 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.471486 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:02:08.471826 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.471809 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" containerName="registry" Jul 15 08:02:08.471896 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.471830 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" containerName="registry" Jul 15 08:02:08.471935 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.471901 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b6f1fa-2e77-4d9d-a48a-1212f1f121e5" containerName="registry" Jul 15 08:02:08.475051 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.475029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.486554 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.486533 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:02:08.576791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.576753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.576791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.576787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw8tw\" (UniqueName: \"kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.577017 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.576813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.577017 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.576887 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.577017 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.576969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.577017 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.577012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.577232 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.577042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.677809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.677772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw8tw\" (UniqueName: \"kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.678551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.678524 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.678692 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.678525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.678692 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.678589 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.678786 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.678689 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.680166 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.680144 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.680336 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.680320 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.685373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.685306 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw8tw\" (UniqueName: \"kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw\") pod \"console-6dfc749f56-gdqjn\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.784440 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.784401 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:08.932854 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:08.932826 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:02:08.936224 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:02:08.936170 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f009a8_1a93_4fea_a71d_d427f7299b80.slice/crio-a61c0a5a73ed24d51ee175c694f33da3799de83ddbf7119b50fb150c56bc8993 WatchSource:0}: Error finding container a61c0a5a73ed24d51ee175c694f33da3799de83ddbf7119b50fb150c56bc8993: Status 404 returned error can't find the container with id a61c0a5a73ed24d51ee175c694f33da3799de83ddbf7119b50fb150c56bc8993 Jul 15 08:02:09.833318 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:09.833273 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc749f56-gdqjn" event={"ID":"06f009a8-1a93-4fea-a71d-d427f7299b80","Type":"ContainerStarted","Data":"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196"} Jul 15 08:02:09.833773 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:09.833340 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc749f56-gdqjn" event={"ID":"06f009a8-1a93-4fea-a71d-d427f7299b80","Type":"ContainerStarted","Data":"a61c0a5a73ed24d51ee175c694f33da3799de83ddbf7119b50fb150c56bc8993"} Jul 15 08:02:09.853740 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:09.853692 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfc749f56-gdqjn" podStartSLOduration=1.8536787609999998 podStartE2EDuration="1.853678761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:02:09.851825468 +0000 UTC m=+93.243230091" watchObservedRunningTime="2026-07-15 08:02:09.853678761 +0000 UTC m=+93.245083361" Jul 15 08:02:14.661114 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:14.661072 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:18.785352 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:18.785306 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:18.785851 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:18.785409 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:18.790193 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:18.790172 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:18.872676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:18.872649 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:02:19.727590 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:19.727556 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nlrjg" Jul 15 08:02:34.852533 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:34.852475 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7648c846ff-srzln" podUID="17331907-69af-4fcd-920b-547d27146b46" containerName="console" containerID="cri-o://11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c" gracePeriod=15 Jul 15 08:02:35.089874 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.089851 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:35.168241 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168165 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168241 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168212 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168241 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168232 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168254 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjc4c\" (UniqueName: \"kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168278 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168391 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168434 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config\") pod \"17331907-69af-4fcd-920b-547d27146b46\" (UID: \"17331907-69af-4fcd-920b-547d27146b46\") " Jul 15 08:02:35.168746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.168722 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:35.169039 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.169013 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config" (OuterVolumeSpecName: "console-config") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:35.169175 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.169036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca" (OuterVolumeSpecName: "service-ca") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:35.169175 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.169108 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:02:35.170961 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.170938 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:02:35.171085 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.171063 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:02:35.171132 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.171082 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c" (OuterVolumeSpecName: "kube-api-access-mjc4c") pod "17331907-69af-4fcd-920b-547d27146b46" (UID: "17331907-69af-4fcd-920b-547d27146b46"). InnerVolumeSpecName "kube-api-access-mjc4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:02:35.269762 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269725 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-trusted-ca-bundle\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269762 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269756 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjc4c\" (UniqueName: \"kubernetes.io/projected/17331907-69af-4fcd-920b-547d27146b46-kube-api-access-mjc4c\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269762 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269772 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-oauth-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269784 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-oauth-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269795 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-console-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269807 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17331907-69af-4fcd-920b-547d27146b46-service-ca\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.269990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.269817 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17331907-69af-4fcd-920b-547d27146b46-console-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:02:35.924445 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.924412 2581 generic.go:356] "Generic (PLEG): container finished" podID="17331907-69af-4fcd-920b-547d27146b46" containerID="11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c" exitCode=0 Jul 15 08:02:35.924838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.924460 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648c846ff-srzln" Jul 15 08:02:35.924838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.924500 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648c846ff-srzln" event={"ID":"17331907-69af-4fcd-920b-547d27146b46","Type":"ContainerDied","Data":"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c"} Jul 15 08:02:35.924838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.924553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648c846ff-srzln" event={"ID":"17331907-69af-4fcd-920b-547d27146b46","Type":"ContainerDied","Data":"883fe0e0606ad8bb14f430927b44288356af795ce33ea078813977d73bdc6299"} Jul 15 08:02:35.924838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.924579 2581 scope.go:117] "RemoveContainer" containerID="11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c" Jul 15 08:02:35.932089 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.932075 2581 scope.go:117] "RemoveContainer" containerID="11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c" Jul 15 08:02:35.932348 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:02:35.932333 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c\": container with ID starting with 11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c not found: ID does not exist" containerID="11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c" Jul 15 08:02:35.932416 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.932375 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c"} err="failed to get container status \"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c\": rpc error: code = NotFound desc = could not find container \"11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c\": container with ID starting with 11d519a4d79aef2a7a72a8d1faaee09f539ae62c85e195eac59e56fcd02d901c not found: ID does not exist" Jul 15 08:02:35.946842 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.946819 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:35.951470 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:35.951449 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7648c846ff-srzln"] Jul 15 08:02:37.354913 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:02:37.354881 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17331907-69af-4fcd-920b-547d27146b46" path="/var/lib/kubelet/pods/17331907-69af-4fcd-920b-547d27146b46/volumes" Jul 15 08:03:27.754459 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.754421 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:03:27.755029 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.754784 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17331907-69af-4fcd-920b-547d27146b46" containerName="console" Jul 15 08:03:27.755029 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.754803 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="17331907-69af-4fcd-920b-547d27146b46" containerName="console" Jul 15 08:03:27.755029 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.754883 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="17331907-69af-4fcd-920b-547d27146b46" containerName="console" Jul 15 08:03:27.756781 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.756758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.784526 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.784496 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:03:27.831461 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksktg\" (UniqueName: \"kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831712 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831712 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.831712 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.831683 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932155 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksktg\" (UniqueName: \"kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932225 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.932307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.933019 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.932994 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.933136 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.933037 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.933136 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.933115 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.933224 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.933133 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.934786 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.934763 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.934876 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.934766 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:27.943977 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:27.943958 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksktg\" (UniqueName: \"kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg\") pod \"console-56485c58df-fljdx\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:28.065380 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:28.065298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:28.199075 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:28.199042 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:03:28.202278 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:03:28.202249 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c210ea9_b4b1_4291_a5b6_5b7b10a6bab5.slice/crio-0ca022aa3c0fdad9891d3fd2ff78aeac9584327a2698f83e4089643719e2284f WatchSource:0}: Error finding container 0ca022aa3c0fdad9891d3fd2ff78aeac9584327a2698f83e4089643719e2284f: Status 404 returned error can't find the container with id 0ca022aa3c0fdad9891d3fd2ff78aeac9584327a2698f83e4089643719e2284f Jul 15 08:03:29.101386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:29.101334 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56485c58df-fljdx" event={"ID":"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5","Type":"ContainerStarted","Data":"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca"} Jul 15 08:03:29.101781 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:29.101421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56485c58df-fljdx" event={"ID":"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5","Type":"ContainerStarted","Data":"0ca022aa3c0fdad9891d3fd2ff78aeac9584327a2698f83e4089643719e2284f"} Jul 15 08:03:29.124327 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:29.124279 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56485c58df-fljdx" podStartSLOduration=2.124264739 podStartE2EDuration="2.124264739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:03:29.12346339 +0000 UTC m=+172.514868013" watchObservedRunningTime="2026-07-15 08:03:29.124264739 +0000 UTC m=+172.515669379" Jul 15 08:03:38.066373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:38.066330 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:38.066754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:38.066402 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:38.071108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:38.071083 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:38.132809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:38.132777 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:03:38.192328 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:03:38.192298 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:04:03.218598 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.218543 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dfc749f56-gdqjn" podUID="06f009a8-1a93-4fea-a71d-d427f7299b80" containerName="console" containerID="cri-o://96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196" gracePeriod=15 Jul 15 08:04:03.459160 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.459138 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:04:03.592465 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592435 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592492 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592539 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592561 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw8tw\" (UniqueName: \"kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592596 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592623 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592867 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592665 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert\") pod \"06f009a8-1a93-4fea-a71d-d427f7299b80\" (UID: \"06f009a8-1a93-4fea-a71d-d427f7299b80\") " Jul 15 08:04:03.592982 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592860 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config" (OuterVolumeSpecName: "console-config") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:04:03.592982 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.592973 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:04:03.593096 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.593040 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca" (OuterVolumeSpecName: "service-ca") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:04:03.593132 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.593086 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:04:03.594716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.594695 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:04:03.594815 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.594752 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw" (OuterVolumeSpecName: "kube-api-access-pw8tw") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "kube-api-access-pw8tw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:04:03.594815 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.594781 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "06f009a8-1a93-4fea-a71d-d427f7299b80" (UID: "06f009a8-1a93-4fea-a71d-d427f7299b80"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:04:03.693720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693685 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-service-ca\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693712 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693721 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-console-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693934 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693730 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-oauth-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693934 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693743 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f009a8-1a93-4fea-a71d-d427f7299b80-console-oauth-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693934 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693753 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw8tw\" (UniqueName: \"kubernetes.io/projected/06f009a8-1a93-4fea-a71d-d427f7299b80-kube-api-access-pw8tw\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:03.693934 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:03.693761 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f009a8-1a93-4fea-a71d-d427f7299b80-trusted-ca-bundle\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:04:04.209682 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.209644 2581 generic.go:356] "Generic (PLEG): container finished" podID="06f009a8-1a93-4fea-a71d-d427f7299b80" containerID="96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196" exitCode=0 Jul 15 08:04:04.209837 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.209687 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfc749f56-gdqjn" Jul 15 08:04:04.209837 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.209724 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc749f56-gdqjn" event={"ID":"06f009a8-1a93-4fea-a71d-d427f7299b80","Type":"ContainerDied","Data":"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196"} Jul 15 08:04:04.209837 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.209770 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfc749f56-gdqjn" event={"ID":"06f009a8-1a93-4fea-a71d-d427f7299b80","Type":"ContainerDied","Data":"a61c0a5a73ed24d51ee175c694f33da3799de83ddbf7119b50fb150c56bc8993"} Jul 15 08:04:04.209837 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.209790 2581 scope.go:117] "RemoveContainer" containerID="96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196" Jul 15 08:04:04.218293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.218277 2581 scope.go:117] "RemoveContainer" containerID="96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196" Jul 15 08:04:04.218584 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:04:04.218563 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196\": container with ID starting with 96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196 not found: ID does not exist" containerID="96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196" Jul 15 08:04:04.218644 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.218592 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196"} err="failed to get container status \"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196\": rpc error: code = NotFound desc = could not find container \"96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196\": container with ID starting with 96a526bb3e7a7742e10746f98059d30bb63bdb66c7c81f775c6a183792543196 not found: ID does not exist" Jul 15 08:04:04.235910 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.235887 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:04:04.239901 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:04.239882 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dfc749f56-gdqjn"] Jul 15 08:04:05.349870 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:05.349837 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f009a8-1a93-4fea-a71d-d427f7299b80" path="/var/lib/kubelet/pods/06f009a8-1a93-4fea-a71d-d427f7299b80/volumes" Jul 15 08:04:53.961161 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.961126 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fj47w"] Jul 15 08:04:53.962154 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.962129 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f009a8-1a93-4fea-a71d-d427f7299b80" containerName="console" Jul 15 08:04:53.962283 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.962273 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f009a8-1a93-4fea-a71d-d427f7299b80" containerName="console" Jul 15 08:04:53.962496 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.962481 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f009a8-1a93-4fea-a71d-d427f7299b80" containerName="console" Jul 15 08:04:53.967204 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.967181 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:53.970856 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.970837 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 08:04:53.971274 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.971247 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 08:04:53.976311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:53.976279 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fj47w"] Jul 15 08:04:54.036374 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.036326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.036533 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.036380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq8ts\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-kube-api-access-bq8ts\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.136972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.136937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.137146 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.136976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq8ts\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-kube-api-access-bq8ts\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.144943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.144920 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.145152 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.145130 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq8ts\" (UniqueName: \"kubernetes.io/projected/6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa-kube-api-access-bq8ts\") pod \"cert-manager-cainjector-559cdd8c87-fj47w\" (UID: \"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.277129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.277036 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" Jul 15 08:04:54.403764 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:54.403735 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fj47w"] Jul 15 08:04:54.406746 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:04:54.406718 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a8ea034_2cdd_4f76_a7f0_f53e3ce091fa.slice/crio-c81a90e02b1437d8a28b874269474796d3a5303d9c09cf841df72b19adac58a5 WatchSource:0}: Error finding container c81a90e02b1437d8a28b874269474796d3a5303d9c09cf841df72b19adac58a5: Status 404 returned error can't find the container with id c81a90e02b1437d8a28b874269474796d3a5303d9c09cf841df72b19adac58a5 Jul 15 08:04:55.379571 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:55.379528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" event={"ID":"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa","Type":"ContainerStarted","Data":"c81a90e02b1437d8a28b874269474796d3a5303d9c09cf841df72b19adac58a5"} Jul 15 08:04:58.392152 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:58.392117 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" event={"ID":"6a8ea034-2cdd-4f76-a7f0-f53e3ce091fa","Type":"ContainerStarted","Data":"61aa9df7843de56c2f893105283157f32dcc5cf2b242281a2bbacb8508a8b666"} Jul 15 08:04:58.409436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:04:58.409348 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fj47w" podStartSLOduration=2.078728078 podStartE2EDuration="5.409333619s" totalImagesPullingTime="3.330605541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:04:58.408841781 +0000 UTC m=+261.800246405" watchObservedRunningTime="2026-07-15 08:04:58.409333619 +0000 UTC m=+261.800738241" Jul 15 08:05:10.859796 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.859760 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv"] Jul 15 08:05:10.862795 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.862777 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:10.865638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.865611 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 08:05:10.865750 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.865695 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 08:05:10.866578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.866563 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vhsbk\"" Jul 15 08:05:10.873350 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.873331 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv"] Jul 15 08:05:10.960089 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.960055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppg5x\" (UniqueName: \"kubernetes.io/projected/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-kube-api-access-ppg5x\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:10.960236 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:10.960099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.061241 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.061206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppg5x\" (UniqueName: \"kubernetes.io/projected/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-kube-api-access-ppg5x\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.061424 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.061259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.061635 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.061616 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.070195 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.070172 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppg5x\" (UniqueName: \"kubernetes.io/projected/3ccae9cf-5aa7-42ad-82ff-e6652e4519e2-kube-api-access-ppg5x\") pod \"openshift-lws-operator-fd8ccff4c-cvstv\" (UID: \"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.171863 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.171779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" Jul 15 08:05:11.292220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.292191 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv"] Jul 15 08:05:11.295054 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:11.295028 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ccae9cf_5aa7_42ad_82ff_e6652e4519e2.slice/crio-611ca272623fcb9bbd2453804f592d2209fbc07ffde70ddb81163583ffb3fdad WatchSource:0}: Error finding container 611ca272623fcb9bbd2453804f592d2209fbc07ffde70ddb81163583ffb3fdad: Status 404 returned error can't find the container with id 611ca272623fcb9bbd2453804f592d2209fbc07ffde70ddb81163583ffb3fdad Jul 15 08:05:11.435525 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:11.435444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" event={"ID":"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2","Type":"ContainerStarted","Data":"611ca272623fcb9bbd2453804f592d2209fbc07ffde70ddb81163583ffb3fdad"} Jul 15 08:05:12.727737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.727701 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5jx5"] Jul 15 08:05:12.731044 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.731013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.733901 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.733878 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-chqg6\"" Jul 15 08:05:12.745238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.745214 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5jx5"] Jul 15 08:05:12.876679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.876646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.876679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.876683 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhbv\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-kube-api-access-5bhbv\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.977721 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.977688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.977721 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.977724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhbv\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-kube-api-access-5bhbv\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.987444 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.987332 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhbv\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-kube-api-access-5bhbv\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:12.987729 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:12.987707 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5e9dfb0-45da-4128-a38b-18f2cd8573da-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5jx5\" (UID: \"d5e9dfb0-45da-4128-a38b-18f2cd8573da\") " pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:13.043425 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:13.043393 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5jx5" Jul 15 08:05:14.019454 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.019426 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5jx5"] Jul 15 08:05:14.020543 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:14.020512 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5e9dfb0_45da_4128_a38b_18f2cd8573da.slice/crio-1940f0278a117a83bc96a5d90878927a2448db54292f7c1c811d98f84a95a7fe WatchSource:0}: Error finding container 1940f0278a117a83bc96a5d90878927a2448db54292f7c1c811d98f84a95a7fe: Status 404 returned error can't find the container with id 1940f0278a117a83bc96a5d90878927a2448db54292f7c1c811d98f84a95a7fe Jul 15 08:05:14.448010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.447977 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" event={"ID":"3ccae9cf-5aa7-42ad-82ff-e6652e4519e2","Type":"ContainerStarted","Data":"5fbd5a253478f17b033cadeccd66ebe7c29ebe25c760726ba45f19b85551866f"} Jul 15 08:05:14.449414 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.449388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5jx5" event={"ID":"d5e9dfb0-45da-4128-a38b-18f2cd8573da","Type":"ContainerStarted","Data":"ecfbf222a99338ecfb24736595f8227d1b29583388e7118bf92b207cb583c3fa"} Jul 15 08:05:14.449554 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.449422 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5jx5" event={"ID":"d5e9dfb0-45da-4128-a38b-18f2cd8573da","Type":"ContainerStarted","Data":"1940f0278a117a83bc96a5d90878927a2448db54292f7c1c811d98f84a95a7fe"} Jul 15 08:05:14.466615 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.466572 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cvstv" podStartSLOduration=1.803031103 podStartE2EDuration="4.466555577s" totalImagesPullingTime="2.663524474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:14.465618971 +0000 UTC m=+277.857023595" watchObservedRunningTime="2026-07-15 08:05:14.466555577 +0000 UTC m=+277.857960203" Jul 15 08:05:14.488868 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:14.488831 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c5jx5" podStartSLOduration=2.488817765 podStartE2EDuration="2.488817765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:14.488711707 +0000 UTC m=+277.880116329" watchObservedRunningTime="2026-07-15 08:05:14.488817765 +0000 UTC m=+277.880222387" Jul 15 08:05:31.472241 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.472202 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d"] Jul 15 08:05:31.474535 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.474516 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.477697 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.477675 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 08:05:31.478045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.478030 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 08:05:31.478285 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.478273 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 08:05:31.478376 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.478337 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5958h\"" Jul 15 08:05:31.478889 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.478869 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 08:05:31.490879 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.490852 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d"] Jul 15 08:05:31.622081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.622038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-manifests\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.622288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.622103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.622288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.622161 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-tmp\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.622288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.622224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.622288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.622255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbfd\" (UniqueName: \"kubernetes.io/projected/ace6dbfa-cf66-47db-916e-bba2a68e9955-kube-api-access-gvbfd\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.722917 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.722826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.722917 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.722867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbfd\" (UniqueName: \"kubernetes.io/projected/ace6dbfa-cf66-47db-916e-bba2a68e9955-kube-api-access-gvbfd\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.722917 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.722913 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-manifests\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.723192 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.723085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.723192 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.723126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-tmp\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.723413 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.723350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-manifests\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.725216 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.725196 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ace6dbfa-cf66-47db-916e-bba2a68e9955-tmp\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.725480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.725460 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.725553 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.725495 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ace6dbfa-cf66-47db-916e-bba2a68e9955-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.735691 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.735660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbfd\" (UniqueName: \"kubernetes.io/projected/ace6dbfa-cf66-47db-916e-bba2a68e9955-kube-api-access-gvbfd\") pod \"opendatahub-operator-controller-manager-7b8c85cff6-k9p7d\" (UID: \"ace6dbfa-cf66-47db-916e-bba2a68e9955\") " pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.785170 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.785126 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:31.949922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:31.949896 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d"] Jul 15 08:05:31.950989 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:31.950958 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace6dbfa_cf66_47db_916e_bba2a68e9955.slice/crio-b1059a1661503b93e5ba60ba857fed7a6ec8ad8c224c09f846b5447f038df9bd WatchSource:0}: Error finding container b1059a1661503b93e5ba60ba857fed7a6ec8ad8c224c09f846b5447f038df9bd: Status 404 returned error can't find the container with id b1059a1661503b93e5ba60ba857fed7a6ec8ad8c224c09f846b5447f038df9bd Jul 15 08:05:32.513805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:32.513765 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" event={"ID":"ace6dbfa-cf66-47db-916e-bba2a68e9955","Type":"ContainerStarted","Data":"b1059a1661503b93e5ba60ba857fed7a6ec8ad8c224c09f846b5447f038df9bd"} Jul 15 08:05:35.530275 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:35.530184 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" event={"ID":"ace6dbfa-cf66-47db-916e-bba2a68e9955","Type":"ContainerStarted","Data":"75f9eb32021f564d4dc829b65f988671d6ed48878bdf66336e9b5ef81444869b"} Jul 15 08:05:36.535682 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:36.535647 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace6dbfa-cf66-47db-916e-bba2a68e9955" containerID="75f9eb32021f564d4dc829b65f988671d6ed48878bdf66336e9b5ef81444869b" exitCode=0 Jul 15 08:05:36.536045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:36.535689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" event={"ID":"ace6dbfa-cf66-47db-916e-bba2a68e9955","Type":"ContainerDied","Data":"75f9eb32021f564d4dc829b65f988671d6ed48878bdf66336e9b5ef81444869b"} Jul 15 08:05:37.055221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.055195 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:05:37.057293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.057269 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 08:05:37.058426 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.058405 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:05:37.542261 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.542228 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" event={"ID":"ace6dbfa-cf66-47db-916e-bba2a68e9955","Type":"ContainerStarted","Data":"f34a707258155b0ec3fcfa050745ab4b629b473489ad1a1ffc839cdc7403eab1"} Jul 15 08:05:37.544474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.542491 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:37.579454 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:37.579401 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" podStartSLOduration=1.8535622520000001 podStartE2EDuration="6.579384722s" totalImagesPullingTime="3.636685187s" totalInitContainerRuntime="1.089137283s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:37.577046056 +0000 UTC m=+300.968450678" watchObservedRunningTime="2026-07-15 08:05:37.579384722 +0000 UTC m=+300.970789341" Jul 15 08:05:48.548429 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:48.548396 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b8c85cff6-k9p7d" Jul 15 08:05:50.830519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.830487 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7"] Jul 15 08:05:50.838081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.838059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:50.841539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.841517 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 08:05:50.841671 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.841596 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 08:05:50.842462 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.842446 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 08:05:50.842547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.842452 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 08:05:50.859903 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.859874 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7"] Jul 15 08:05:50.981689 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.981656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tls-certs\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:50.981689 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.981688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tmp\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:50.981885 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:50.981716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwftm\" (UniqueName: \"kubernetes.io/projected/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-kube-api-access-zwftm\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.083089 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.083065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tls-certs\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.083213 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.083096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tmp\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.083213 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.083117 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwftm\" (UniqueName: \"kubernetes.io/projected/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-kube-api-access-zwftm\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.085534 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.085510 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tmp\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.085644 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.085612 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-tls-certs\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.110530 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.110505 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwftm\" (UniqueName: \"kubernetes.io/projected/49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2-kube-api-access-zwftm\") pod \"kube-auth-proxy-657b79bd8d-9dqm7\" (UID: \"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2\") " pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.147352 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.147324 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" Jul 15 08:05:51.310187 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.310164 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7"] Jul 15 08:05:51.311748 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:51.311724 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e34b7e_2ad0_4fa3_9e1f_abc59c119fe2.slice/crio-ca26569798446f671af133fdc51f13439600d1173f3692408638a5438c363e7e WatchSource:0}: Error finding container ca26569798446f671af133fdc51f13439600d1173f3692408638a5438c363e7e: Status 404 returned error can't find the container with id ca26569798446f671af133fdc51f13439600d1173f3692408638a5438c363e7e Jul 15 08:05:51.313483 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.313464 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 08:05:51.541268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.541190 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-25n98"] Jul 15 08:05:51.544480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.544462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.547322 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.547304 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 08:05:51.547807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.547789 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-c5vfz\"" Jul 15 08:05:51.577709 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.577687 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-25n98"] Jul 15 08:05:51.603633 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.603609 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" event={"ID":"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2","Type":"ContainerStarted","Data":"ca26569798446f671af133fdc51f13439600d1173f3692408638a5438c363e7e"} Jul 15 08:05:51.687315 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.687287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7x4x\" (UniqueName: \"kubernetes.io/projected/65403957-a5d6-48c7-aa23-f2a8fb8b1319-kube-api-access-h7x4x\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.687484 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.687342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65403957-a5d6-48c7-aa23-f2a8fb8b1319-cert\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.787958 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.787912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65403957-a5d6-48c7-aa23-f2a8fb8b1319-cert\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.788113 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.787992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7x4x\" (UniqueName: \"kubernetes.io/projected/65403957-a5d6-48c7-aa23-f2a8fb8b1319-kube-api-access-h7x4x\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.790254 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.790234 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65403957-a5d6-48c7-aa23-f2a8fb8b1319-cert\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.801749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.801697 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7x4x\" (UniqueName: \"kubernetes.io/projected/65403957-a5d6-48c7-aa23-f2a8fb8b1319-kube-api-access-h7x4x\") pod \"odh-model-controller-7998c67dc4-25n98\" (UID: \"65403957-a5d6-48c7-aa23-f2a8fb8b1319\") " pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.854228 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.854203 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:51.984686 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:51.984655 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-25n98"] Jul 15 08:05:51.987613 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:51.987590 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65403957_a5d6_48c7_aa23_f2a8fb8b1319.slice/crio-aec749a43f20511ded1a4e4ad86ae6acf0a162f483fdfdbfef7cd8d0d57aeab4 WatchSource:0}: Error finding container aec749a43f20511ded1a4e4ad86ae6acf0a162f483fdfdbfef7cd8d0d57aeab4: Status 404 returned error can't find the container with id aec749a43f20511ded1a4e4ad86ae6acf0a162f483fdfdbfef7cd8d0d57aeab4 Jul 15 08:05:52.609735 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.609674 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" event={"ID":"65403957-a5d6-48c7-aa23-f2a8fb8b1319","Type":"ContainerStarted","Data":"aec749a43f20511ded1a4e4ad86ae6acf0a162f483fdfdbfef7cd8d0d57aeab4"} Jul 15 08:05:52.658751 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.658718 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jbwq8"] Jul 15 08:05:52.661396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.661368 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.663954 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.663933 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 08:05:52.664082 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.664031 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8gwht\"" Jul 15 08:05:52.675200 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.675168 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jbwq8"] Jul 15 08:05:52.791596 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.791559 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:05:52.794405 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.794378 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.795155 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.795012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.795155 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.795097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j8gk\" (UniqueName: \"kubernetes.io/projected/b98ee470-8fb3-47a7-bfc2-ec216305b98f-kube-api-access-6j8gk\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.797210 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.797188 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 08:05:52.797210 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.797201 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 08:05:52.797406 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.797188 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 08:05:52.797406 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.797331 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b6j64\"" Jul 15 08:05:52.797599 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.797573 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 08:05:52.812661 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.812637 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:05:52.896023 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.895939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.896023 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.895989 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896023 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:05:52.896114 2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896173 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:05:52.896180 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs podName:b98ee470-8fb3-47a7-bfc2-ec216305b98f nodeName:}" failed. No retries permitted until 2026-07-15 08:05:53.396160215 +0000 UTC m=+316.787564816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs") pod "model-serving-api-54d89bd5fc-jbwq8" (UID: "b98ee470-8fb3-47a7-bfc2-ec216305b98f") : secret "model-serving-api-tls" not found Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896218 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896429 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-229vs\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.896830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j8gk\" (UniqueName: \"kubernetes.io/projected/b98ee470-8fb3-47a7-bfc2-ec216305b98f-kube-api-access-6j8gk\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.896830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.896563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.905958 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.905935 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j8gk\" (UniqueName: \"kubernetes.io/projected/b98ee470-8fb3-47a7-bfc2-ec216305b98f-kube-api-access-6j8gk\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:52.997615 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.997790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.997790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.997790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997712 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.997790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.997790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997791 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-229vs\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.998073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.997818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:52.998629 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:52.998583 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.000623 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.000594 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.000726 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.000632 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.000726 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.000675 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.000835 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.000741 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.007096 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.007059 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.007481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.007436 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-229vs\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs\") pod \"istiod-openshift-gateway-7574459656-g4zhv\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.105416 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.105379 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:53.266846 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.266812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:05:53.269532 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:53.269501 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0c8fa6_6fdd_478a_a80f_89bddde22919.slice/crio-54207b69ebcb00589daf54a8d7260d61247e0d691ee4ddbf12330e32022c54de WatchSource:0}: Error finding container 54207b69ebcb00589daf54a8d7260d61247e0d691ee4ddbf12330e32022c54de: Status 404 returned error can't find the container with id 54207b69ebcb00589daf54a8d7260d61247e0d691ee4ddbf12330e32022c54de Jul 15 08:05:53.402167 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.401662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:53.405515 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.405465 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b98ee470-8fb3-47a7-bfc2-ec216305b98f-tls-certs\") pod \"model-serving-api-54d89bd5fc-jbwq8\" (UID: \"b98ee470-8fb3-47a7-bfc2-ec216305b98f\") " pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:53.572277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.572199 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:53.616975 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.616894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" event={"ID":"1c0c8fa6-6fdd-478a-a80f-89bddde22919","Type":"ContainerStarted","Data":"54207b69ebcb00589daf54a8d7260d61247e0d691ee4ddbf12330e32022c54de"} Jul 15 08:05:53.736260 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:53.736225 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jbwq8"] Jul 15 08:05:53.739627 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:53.739582 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98ee470_8fb3_47a7_bfc2_ec216305b98f.slice/crio-e9def1bbb8c0e36d7ed8442741d3ab0c5b60f8a5d2204e06305b694d525c4e93 WatchSource:0}: Error finding container e9def1bbb8c0e36d7ed8442741d3ab0c5b60f8a5d2204e06305b694d525c4e93: Status 404 returned error can't find the container with id e9def1bbb8c0e36d7ed8442741d3ab0c5b60f8a5d2204e06305b694d525c4e93 Jul 15 08:05:54.627387 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:54.622862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" event={"ID":"b98ee470-8fb3-47a7-bfc2-ec216305b98f","Type":"ContainerStarted","Data":"e9def1bbb8c0e36d7ed8442741d3ab0c5b60f8a5d2204e06305b694d525c4e93"} Jul 15 08:05:56.636452 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:56.636413 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" event={"ID":"65403957-a5d6-48c7-aa23-f2a8fb8b1319","Type":"ContainerStarted","Data":"44d358d782c839f260acccdd66bc4048c39fd8afd5f9b6d2947d41d9ecf121f5"} Jul 15 08:05:56.636918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:56.636596 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:05:56.638024 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:56.637993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" event={"ID":"49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2","Type":"ContainerStarted","Data":"0ab3fac2e5894521343b9baf8c20d64714895c72ec39287e0c8507e8811ef76c"} Jul 15 08:05:56.653917 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:56.653864 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" podStartSLOduration=1.9572028399999999 podStartE2EDuration="5.653851048s" totalImagesPullingTime="3.696648208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:56.653464629 +0000 UTC m=+320.044869253" watchObservedRunningTime="2026-07-15 08:05:56.653851048 +0000 UTC m=+320.045255671" Jul 15 08:05:56.669956 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:56.669905 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-657b79bd8d-9dqm7" podStartSLOduration=2.299944518 podStartE2EDuration="6.669886997s" totalImagesPullingTime="4.369942479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:56.668795197 +0000 UTC m=+320.060199840" watchObservedRunningTime="2026-07-15 08:05:56.669886997 +0000 UTC m=+320.061291623" Jul 15 08:05:57.631430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:57.631315 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:05:57.631430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:57.631390 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:05:58.648827 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.648787 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" event={"ID":"1c0c8fa6-6fdd-478a-a80f-89bddde22919","Type":"ContainerStarted","Data":"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f"} Jul 15 08:05:58.649240 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.648949 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:05:58.650435 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.650397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" event={"ID":"b98ee470-8fb3-47a7-bfc2-ec216305b98f","Type":"ContainerStarted","Data":"b70bf1604cbe1d1748103cd81c3b93ee84e6b2b66accb00998bde69cbb38169e"} Jul 15 08:05:58.650575 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.650546 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:05:58.650797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.650772 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-g4zhv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:05:58.650886 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.650820 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:05:58.678875 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.678833 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" podStartSLOduration=2.319780213 podStartE2EDuration="6.678819771s" totalImagesPullingTime="4.359039558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:58.677763204 +0000 UTC m=+322.069167854" watchObservedRunningTime="2026-07-15 08:05:58.678819771 +0000 UTC m=+322.070224394" Jul 15 08:05:58.696638 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.696586 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" podStartSLOduration=2.812282398 podStartE2EDuration="6.696570742s" totalImagesPullingTime="3.884288344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:05:58.695341964 +0000 UTC m=+322.086746606" watchObservedRunningTime="2026-07-15 08:05:58.696570742 +0000 UTC m=+322.087975364" Jul 15 08:05:58.976158 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.976080 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6"] Jul 15 08:05:58.978347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.978326 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:58.982978 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:58.982955 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mqgw8\"" Jul 15 08:05:59.005059 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.005036 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6"] Jul 15 08:05:59.048567 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048576 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7mnx\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-kube-api-access-z7mnx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048603 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048720 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6929599-b919-4016-ab37-82dd9d3604c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048919 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048919 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048843 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6929599-b919-4016-ab37-82dd9d3604c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.048992 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.048911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7mnx\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-kube-api-access-z7mnx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149833 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6929599-b919-4016-ab37-82dd9d3604c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.149922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150215 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.149920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6929599-b919-4016-ab37-82dd9d3604c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150215 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150203 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150289 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150457 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150428 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150542 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.150809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.150789 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6929599-b919-4016-ab37-82dd9d3604c3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.152566 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.152543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6929599-b919-4016-ab37-82dd9d3604c3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.152756 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.152739 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6929599-b919-4016-ab37-82dd9d3604c3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.168509 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.168488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7mnx\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-kube-api-access-z7mnx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.168937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.168917 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6929599-b919-4016-ab37-82dd9d3604c3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gdmx6\" (UID: \"c6929599-b919-4016-ab37-82dd9d3604c3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.288842 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.288746 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:05:59.476805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.476778 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6"] Jul 15 08:05:59.478473 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:05:59.478442 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6929599_b919_4016_ab37_82dd9d3604c3.slice/crio-2538342b4ed081e75911c6b6c6a0b0b377f3715692c3bc8e3b0a609ebd86f0f8 WatchSource:0}: Error finding container 2538342b4ed081e75911c6b6c6a0b0b377f3715692c3bc8e3b0a609ebd86f0f8: Status 404 returned error can't find the container with id 2538342b4ed081e75911c6b6c6a0b0b377f3715692c3bc8e3b0a609ebd86f0f8 Jul 15 08:05:59.656540 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.656501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" event={"ID":"c6929599-b919-4016-ab37-82dd9d3604c3","Type":"ContainerStarted","Data":"2538342b4ed081e75911c6b6c6a0b0b377f3715692c3bc8e3b0a609ebd86f0f8"} Jul 15 08:05:59.657256 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:05:59.657240 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:06:01.988498 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:01.988461 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l"] Jul 15 08:06:01.997546 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:01.997078 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:01.999458 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:01.999431 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4"] Jul 15 08:06:02.000392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.000343 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 08:06:02.000698 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.000681 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-whjrl\"" Jul 15 08:06:02.003417 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.003396 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.006797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.006776 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dkjbb\"" Jul 15 08:06:02.007040 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.007021 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 08:06:02.016097 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.016071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l"] Jul 15 08:06:02.037972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.037937 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4"] Jul 15 08:06:02.080953 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.080905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.080953 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.080956 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca98520-8c57-4354-a1e5-c59716fbd813-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.081197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.081044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twthx\" (UniqueName: \"kubernetes.io/projected/29bff1dd-e66f-4564-8e95-8a09fd61e029-kube-api-access-twthx\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.081197 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.081097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpds9\" (UniqueName: \"kubernetes.io/projected/aca98520-8c57-4354-a1e5-c59716fbd813-kube-api-access-cpds9\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.178636 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.178597 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:06:02.178759 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.178674 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:06:02.178759 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.178708 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:06:02.181812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.181787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twthx\" (UniqueName: \"kubernetes.io/projected/29bff1dd-e66f-4564-8e95-8a09fd61e029-kube-api-access-twthx\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.181937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.181830 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpds9\" (UniqueName: \"kubernetes.io/projected/aca98520-8c57-4354-a1e5-c59716fbd813-kube-api-access-cpds9\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.181937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.181879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.181937 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.181902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca98520-8c57-4354-a1e5-c59716fbd813-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.182102 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:06:02.181992 2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 15 08:06:02.182102 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:06:02.182042 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert podName:29bff1dd-e66f-4564-8e95-8a09fd61e029 nodeName:}" failed. No retries permitted until 2026-07-15 08:06:02.682027894 +0000 UTC m=+326.073432494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert") pod "kserve-controller-manager-d6f4b65b8-wlm7l" (UID: "29bff1dd-e66f-4564-8e95-8a09fd61e029") : secret "kserve-webhook-server-cert" not found Jul 15 08:06:02.184311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.184293 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca98520-8c57-4354-a1e5-c59716fbd813-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.203996 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.203971 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twthx\" (UniqueName: \"kubernetes.io/projected/29bff1dd-e66f-4564-8e95-8a09fd61e029-kube-api-access-twthx\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.204475 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.204455 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpds9\" (UniqueName: \"kubernetes.io/projected/aca98520-8c57-4354-a1e5-c59716fbd813-kube-api-access-cpds9\") pod \"llmisvc-controller-manager-777c6bd7d5-kzvf4\" (UID: \"aca98520-8c57-4354-a1e5-c59716fbd813\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.322522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.322488 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:02.474135 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.474099 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4"] Jul 15 08:06:02.477779 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:06:02.477751 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaca98520_8c57_4354_a1e5_c59716fbd813.slice/crio-0be7691e6aab6d6d55a03ce3115b6dc1b2f3e76f824a744caf161f165fed94de WatchSource:0}: Error finding container 0be7691e6aab6d6d55a03ce3115b6dc1b2f3e76f824a744caf161f165fed94de: Status 404 returned error can't find the container with id 0be7691e6aab6d6d55a03ce3115b6dc1b2f3e76f824a744caf161f165fed94de Jul 15 08:06:02.672721 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.672684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" event={"ID":"c6929599-b919-4016-ab37-82dd9d3604c3","Type":"ContainerStarted","Data":"4d1a59d7d6ca57ce6d38ea482193edcedf97f09080607abc3b24a59e27adc6fc"} Jul 15 08:06:02.673769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.673734 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" event={"ID":"aca98520-8c57-4354-a1e5-c59716fbd813","Type":"ContainerStarted","Data":"0be7691e6aab6d6d55a03ce3115b6dc1b2f3e76f824a744caf161f165fed94de"} Jul 15 08:06:02.685668 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.685644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.688033 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.688004 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29bff1dd-e66f-4564-8e95-8a09fd61e029-cert\") pod \"kserve-controller-manager-d6f4b65b8-wlm7l\" (UID: \"29bff1dd-e66f-4564-8e95-8a09fd61e029\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:02.695246 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.695200 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" podStartSLOduration=1.997822047 podStartE2EDuration="4.695184836s" totalImagesPullingTime="2.697362789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:06:02.694423311 +0000 UTC m=+326.085827959" watchObservedRunningTime="2026-07-15 08:06:02.695184836 +0000 UTC m=+326.086589459" Jul 15 08:06:02.775582 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.775541 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:06:02.775756 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.775712 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerName="discovery" containerID="cri-o://6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f" gracePeriod=30 Jul 15 08:06:02.915026 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:02.914990 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:03.015316 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.015287 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:06:03.067087 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.067058 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l"] Jul 15 08:06:03.069243 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:06:03.069209 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bff1dd_e66f_4564_8e95_8a09fd61e029.slice/crio-ed6d8f0a7f3af9501a48ddd894ff85f63a0bd77b2af9a42373d60f9aeaa6f526 WatchSource:0}: Error finding container ed6d8f0a7f3af9501a48ddd894ff85f63a0bd77b2af9a42373d60f9aeaa6f526: Status 404 returned error can't find the container with id ed6d8f0a7f3af9501a48ddd894ff85f63a0bd77b2af9a42373d60f9aeaa6f526 Jul 15 08:06:03.089524 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.089481 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.089709 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.089538 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.089709 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.089583 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.089709 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.089623 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-229vs\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.090893 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.090015 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.090893 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.090098 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.090893 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.090193 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap\") pod \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\" (UID: \"1c0c8fa6-6fdd-478a-a80f-89bddde22919\") " Jul 15 08:06:03.098613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.091212 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:06:03.098613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.092801 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs" (OuterVolumeSpecName: "local-certs") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:06:03.098613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.094079 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:06:03.098613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.096526 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:06:03.098833 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.098686 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:06:03.110442 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.103501 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs" (OuterVolumeSpecName: "kube-api-access-229vs") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "kube-api-access-229vs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:06:03.110442 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.103647 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts" (OuterVolumeSpecName: "cacerts") pod "1c0c8fa6-6fdd-478a-a80f-89bddde22919" (UID: "1c0c8fa6-6fdd-478a-a80f-89bddde22919"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192506 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-dns-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192540 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c0c8fa6-6fdd-478a-a80f-89bddde22919-local-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192556 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-229vs\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-kube-api-access-229vs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192571 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-cacerts\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192583 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192597 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-csr-ca-configmap\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.192873 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.192609 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c0c8fa6-6fdd-478a-a80f-89bddde22919-istio-kubeconfig\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:03.289162 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.289123 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:06:03.294417 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.294218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:06:03.679563 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.679528 2581 generic.go:356] "Generic (PLEG): container finished" podID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerID="6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f" exitCode=0 Jul 15 08:06:03.679767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.679609 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" Jul 15 08:06:03.679767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.679643 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" event={"ID":"1c0c8fa6-6fdd-478a-a80f-89bddde22919","Type":"ContainerDied","Data":"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f"} Jul 15 08:06:03.679767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.679687 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv" event={"ID":"1c0c8fa6-6fdd-478a-a80f-89bddde22919","Type":"ContainerDied","Data":"54207b69ebcb00589daf54a8d7260d61247e0d691ee4ddbf12330e32022c54de"} Jul 15 08:06:03.679767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.679713 2581 scope.go:117] "RemoveContainer" containerID="6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f" Jul 15 08:06:03.682471 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.682444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" event={"ID":"29bff1dd-e66f-4564-8e95-8a09fd61e029","Type":"ContainerStarted","Data":"ed6d8f0a7f3af9501a48ddd894ff85f63a0bd77b2af9a42373d60f9aeaa6f526"} Jul 15 08:06:03.682592 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.682504 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:06:03.683623 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.683602 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gdmx6" Jul 15 08:06:03.690277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.690250 2581 scope.go:117] "RemoveContainer" containerID="6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f" Jul 15 08:06:03.690792 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:06:03.690764 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f\": container with ID starting with 6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f not found: ID does not exist" containerID="6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f" Jul 15 08:06:03.690874 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.690802 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f"} err="failed to get container status \"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f\": rpc error: code = NotFound desc = could not find container \"6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f\": container with ID starting with 6539128c09379e87b24e2894a661d6499e1cee694a95398b64e5558934bed66f not found: ID does not exist" Jul 15 08:06:03.698598 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.698566 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:06:03.704621 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:03.704594 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-g4zhv"] Jul 15 08:06:05.352423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:05.352380 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" path="/var/lib/kubelet/pods/1c0c8fa6-6fdd-478a-a80f-89bddde22919/volumes" Jul 15 08:06:06.696952 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.696912 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" event={"ID":"29bff1dd-e66f-4564-8e95-8a09fd61e029","Type":"ContainerStarted","Data":"32fb4dd0b831de6966999f86223cf19e8f325746e6109b40809c6836766a8bdd"} Jul 15 08:06:06.697440 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.696987 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:06.698397 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.698373 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" event={"ID":"aca98520-8c57-4354-a1e5-c59716fbd813","Type":"ContainerStarted","Data":"15635ab3f6a9685ac042a92787b25c71b6d3e1eec29abfa1c5e1d5a1e66c7ca9"} Jul 15 08:06:06.698531 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.698510 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:06.716251 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.716196 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" podStartSLOduration=2.515120073 podStartE2EDuration="5.716177289s" totalImagesPullingTime="3.201057216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:06:06.715045071 +0000 UTC m=+330.106449695" watchObservedRunningTime="2026-07-15 08:06:06.716177289 +0000 UTC m=+330.107581912" Jul 15 08:06:06.737794 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:06.737744 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" podStartSLOduration=1.941053315 podStartE2EDuration="5.73772654s" totalImagesPullingTime="3.796673225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:06:06.735990525 +0000 UTC m=+330.127395169" watchObservedRunningTime="2026-07-15 08:06:06.73772654 +0000 UTC m=+330.129131163" Jul 15 08:06:07.645064 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:07.645027 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-25n98" Jul 15 08:06:09.661314 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:09.661284 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jbwq8" Jul 15 08:06:37.706615 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:37.706581 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kzvf4" Jul 15 08:06:37.709566 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:37.709541 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wlm7l" Jul 15 08:06:51.884429 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.884389 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:51.884818 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.884728 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerName="discovery" Jul 15 08:06:51.884818 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.884742 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerName="discovery" Jul 15 08:06:51.884818 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.884786 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c0c8fa6-6fdd-478a-a80f-89bddde22919" containerName="discovery" Jul 15 08:06:51.892749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.892723 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:51.895798 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.895460 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 08:06:51.895798 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.895576 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 08:06:51.896292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.896271 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ckwgh\"" Jul 15 08:06:51.897411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.897385 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:51.917847 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:51.917816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdq6q\" (UniqueName: \"kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q\") pod \"kuadrant-operator-catalog-xdz7j\" (UID: \"8547a889-d39f-41aa-8655-41f86bf20139\") " pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:52.018575 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.018536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdq6q\" (UniqueName: \"kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q\") pod \"kuadrant-operator-catalog-xdz7j\" (UID: \"8547a889-d39f-41aa-8655-41f86bf20139\") " pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:52.027894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.027863 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdq6q\" (UniqueName: \"kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q\") pod \"kuadrant-operator-catalog-xdz7j\" (UID: \"8547a889-d39f-41aa-8655-41f86bf20139\") " pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:52.203920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.203829 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:52.247262 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.247229 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:52.335320 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.335283 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:52.339736 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:06:52.339708 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8547a889_d39f_41aa_8655_41f86bf20139.slice/crio-647a8407deada5f4982bace226123e226a5ac8d48c359ec9eebc20c6db268dfd WatchSource:0}: Error finding container 647a8407deada5f4982bace226123e226a5ac8d48c359ec9eebc20c6db268dfd: Status 404 returned error can't find the container with id 647a8407deada5f4982bace226123e226a5ac8d48c359ec9eebc20c6db268dfd Jul 15 08:06:52.459057 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.458977 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcl55"] Jul 15 08:06:52.461877 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.461857 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:06:52.470282 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.470258 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcl55"] Jul 15 08:06:52.523062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.523025 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd7df\" (UniqueName: \"kubernetes.io/projected/f932eb95-29ec-498a-89d8-8c4d9f9d3066-kube-api-access-nd7df\") pod \"kuadrant-operator-catalog-pcl55\" (UID: \"f932eb95-29ec-498a-89d8-8c4d9f9d3066\") " pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:06:52.623732 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.623692 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd7df\" (UniqueName: \"kubernetes.io/projected/f932eb95-29ec-498a-89d8-8c4d9f9d3066-kube-api-access-nd7df\") pod \"kuadrant-operator-catalog-pcl55\" (UID: \"f932eb95-29ec-498a-89d8-8c4d9f9d3066\") " pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:06:52.634095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.634067 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd7df\" (UniqueName: \"kubernetes.io/projected/f932eb95-29ec-498a-89d8-8c4d9f9d3066-kube-api-access-nd7df\") pod \"kuadrant-operator-catalog-pcl55\" (UID: \"f932eb95-29ec-498a-89d8-8c4d9f9d3066\") " pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:06:52.771730 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.771637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:06:52.892500 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.892462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" event={"ID":"8547a889-d39f-41aa-8655-41f86bf20139","Type":"ContainerStarted","Data":"647a8407deada5f4982bace226123e226a5ac8d48c359ec9eebc20c6db268dfd"} Jul 15 08:06:52.902199 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:52.902167 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pcl55"] Jul 15 08:06:52.905432 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:06:52.905409 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf932eb95_29ec_498a_89d8_8c4d9f9d3066.slice/crio-1117c2c88a3864b4dedead04b765481df10240ff2109de928899c21c44f251db WatchSource:0}: Error finding container 1117c2c88a3864b4dedead04b765481df10240ff2109de928899c21c44f251db: Status 404 returned error can't find the container with id 1117c2c88a3864b4dedead04b765481df10240ff2109de928899c21c44f251db Jul 15 08:06:53.898316 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:53.898283 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" event={"ID":"f932eb95-29ec-498a-89d8-8c4d9f9d3066","Type":"ContainerStarted","Data":"1117c2c88a3864b4dedead04b765481df10240ff2109de928899c21c44f251db"} Jul 15 08:06:55.908352 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:55.908316 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" event={"ID":"f932eb95-29ec-498a-89d8-8c4d9f9d3066","Type":"ContainerStarted","Data":"8a733e2ce9746317ab5918f36abb35ed88439bc034cb79c1861cbbb6af665703"} Jul 15 08:06:55.909839 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:55.909816 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" event={"ID":"8547a889-d39f-41aa-8655-41f86bf20139","Type":"ContainerStarted","Data":"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a"} Jul 15 08:06:55.909922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:55.909899 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" podUID="8547a889-d39f-41aa-8655-41f86bf20139" containerName="registry-server" containerID="cri-o://de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a" gracePeriod=2 Jul 15 08:06:55.928754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:55.928715 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" podStartSLOduration=1.5133382050000002 podStartE2EDuration="3.928700165s" totalImagesPullingTime="2.41536196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:06:55.927850121 +0000 UTC m=+379.319254744" watchObservedRunningTime="2026-07-15 08:06:55.928700165 +0000 UTC m=+379.320104788" Jul 15 08:06:55.946679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:55.946634 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" podStartSLOduration=1.962948396 podStartE2EDuration="4.946618608s" totalImagesPullingTime="2.983670212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:06:55.945501917 +0000 UTC m=+379.336906541" watchObservedRunningTime="2026-07-15 08:06:55.946618608 +0000 UTC m=+379.338023230" Jul 15 08:06:56.149012 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.148989 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:56.258878 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.258782 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdq6q\" (UniqueName: \"kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q\") pod \"8547a889-d39f-41aa-8655-41f86bf20139\" (UID: \"8547a889-d39f-41aa-8655-41f86bf20139\") " Jul 15 08:06:56.261015 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.260984 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q" (OuterVolumeSpecName: "kube-api-access-cdq6q") pod "8547a889-d39f-41aa-8655-41f86bf20139" (UID: "8547a889-d39f-41aa-8655-41f86bf20139"). InnerVolumeSpecName "kube-api-access-cdq6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:06:56.360218 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.360181 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdq6q\" (UniqueName: \"kubernetes.io/projected/8547a889-d39f-41aa-8655-41f86bf20139-kube-api-access-cdq6q\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:06:56.914746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.914709 2581 generic.go:356] "Generic (PLEG): container finished" podID="8547a889-d39f-41aa-8655-41f86bf20139" containerID="de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a" exitCode=0 Jul 15 08:06:56.915221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.914750 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" Jul 15 08:06:56.915221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.914801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" event={"ID":"8547a889-d39f-41aa-8655-41f86bf20139","Type":"ContainerDied","Data":"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a"} Jul 15 08:06:56.915221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.914861 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xdz7j" event={"ID":"8547a889-d39f-41aa-8655-41f86bf20139","Type":"ContainerDied","Data":"647a8407deada5f4982bace226123e226a5ac8d48c359ec9eebc20c6db268dfd"} Jul 15 08:06:56.915221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.914894 2581 scope.go:117] "RemoveContainer" containerID="de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a" Jul 15 08:06:56.924406 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.924381 2581 scope.go:117] "RemoveContainer" containerID="de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a" Jul 15 08:06:56.924690 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:06:56.924671 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a\": container with ID starting with de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a not found: ID does not exist" containerID="de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a" Jul 15 08:06:56.924741 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.924700 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a"} err="failed to get container status \"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a\": rpc error: code = NotFound desc = could not find container \"de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a\": container with ID starting with de4c64ae3261326a53f850d469725e97b9bb675422c306bc3fac067d4f71f10a not found: ID does not exist" Jul 15 08:06:56.938867 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.938822 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:56.940578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:56.940554 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xdz7j"] Jul 15 08:06:57.352010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:06:57.351975 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8547a889-d39f-41aa-8655-41f86bf20139" path="/var/lib/kubelet/pods/8547a889-d39f-41aa-8655-41f86bf20139/volumes" Jul 15 08:07:02.771894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:02.771863 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:07:02.772343 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:02.771917 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:07:02.794617 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:02.794584 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:07:02.961761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:02.961732 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pcl55" Jul 15 08:07:15.991091 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:15.991009 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6df465f9dd-gxsk8"] Jul 15 08:07:15.991497 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:15.991319 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8547a889-d39f-41aa-8655-41f86bf20139" containerName="registry-server" Jul 15 08:07:15.991497 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:15.991329 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8547a889-d39f-41aa-8655-41f86bf20139" containerName="registry-server" Jul 15 08:07:15.991497 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:15.991398 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8547a889-d39f-41aa-8655-41f86bf20139" containerName="registry-server" Jul 15 08:07:15.993616 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:15.993596 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.004670 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.004643 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6df465f9dd-gxsk8"] Jul 15 08:07:16.133583 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133583 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-oauth-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-oauth-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133723 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-trusted-ca-bundle\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133809 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133805 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-service-ca\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.133932 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.133822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9jjl\" (UniqueName: \"kubernetes.io/projected/233afcd7-5d37-4d8a-a657-5e96b12150b9-kube-api-access-f9jjl\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.234995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.234960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-trusted-ca-bundle\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.234995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-service-ca\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235172 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jjl\" (UniqueName: \"kubernetes.io/projected/233afcd7-5d37-4d8a-a657-5e96b12150b9-kube-api-access-f9jjl\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235172 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235172 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-oauth-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-oauth-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235342 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235873 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235846 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-service-ca\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235998 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235893 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-oauth-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235998 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235947 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.235998 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.235965 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233afcd7-5d37-4d8a-a657-5e96b12150b9-trusted-ca-bundle\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.237666 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.237636 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-oauth-config\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.237886 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.237864 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233afcd7-5d37-4d8a-a657-5e96b12150b9-console-serving-cert\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.243905 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.243861 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9jjl\" (UniqueName: \"kubernetes.io/projected/233afcd7-5d37-4d8a-a657-5e96b12150b9-kube-api-access-f9jjl\") pod \"console-6df465f9dd-gxsk8\" (UID: \"233afcd7-5d37-4d8a-a657-5e96b12150b9\") " pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.303676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.303637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:16.435661 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:16.435636 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6df465f9dd-gxsk8"] Jul 15 08:07:16.437814 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:16.437783 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233afcd7_5d37_4d8a_a657_5e96b12150b9.slice/crio-ce70b0754743307fc24dcb9337757855b12b29f86c24e4e9108d37faa81948a6 WatchSource:0}: Error finding container ce70b0754743307fc24dcb9337757855b12b29f86c24e4e9108d37faa81948a6: Status 404 returned error can't find the container with id ce70b0754743307fc24dcb9337757855b12b29f86c24e4e9108d37faa81948a6 Jul 15 08:07:17.001830 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:17.001794 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df465f9dd-gxsk8" event={"ID":"233afcd7-5d37-4d8a-a657-5e96b12150b9","Type":"ContainerStarted","Data":"32c37e0269c996ca717c05cfc930b5aa3a5c6717707ccc94036e7b10fd75786b"} Jul 15 08:07:17.002234 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:17.001842 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6df465f9dd-gxsk8" event={"ID":"233afcd7-5d37-4d8a-a657-5e96b12150b9","Type":"ContainerStarted","Data":"ce70b0754743307fc24dcb9337757855b12b29f86c24e4e9108d37faa81948a6"} Jul 15 08:07:17.033589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:17.033535 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6df465f9dd-gxsk8" podStartSLOduration=2.033517367 podStartE2EDuration="2.033517367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:17.031031344 +0000 UTC m=+400.422436079" watchObservedRunningTime="2026-07-15 08:07:17.033517367 +0000 UTC m=+400.424921989" Jul 15 08:07:23.345736 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.345699 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh"] Jul 15 08:07:23.350048 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.350023 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:23.353181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.353158 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 08:07:23.353305 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.353188 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rw4w9\"" Jul 15 08:07:23.361338 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.361314 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh"] Jul 15 08:07:23.501649 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.501613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x9hn\" (UniqueName: \"kubernetes.io/projected/a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a-kube-api-access-9x9hn\") pod \"dns-operator-controller-manager-864dd78c4c-4cwxh\" (UID: \"a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:23.602926 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.602835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9hn\" (UniqueName: \"kubernetes.io/projected/a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a-kube-api-access-9x9hn\") pod \"dns-operator-controller-manager-864dd78c4c-4cwxh\" (UID: \"a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:23.613652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.613623 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x9hn\" (UniqueName: \"kubernetes.io/projected/a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a-kube-api-access-9x9hn\") pod \"dns-operator-controller-manager-864dd78c4c-4cwxh\" (UID: \"a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:23.660568 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.660530 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:23.791278 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:23.791250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh"] Jul 15 08:07:23.794604 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:23.794571 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45e74a8_6f65_4d0c_9baf_9fba01f7ae0a.slice/crio-889d26760e51c5fbb78555b3fcdf116e9605db235772553b172a4eccd34acdc9 WatchSource:0}: Error finding container 889d26760e51c5fbb78555b3fcdf116e9605db235772553b172a4eccd34acdc9: Status 404 returned error can't find the container with id 889d26760e51c5fbb78555b3fcdf116e9605db235772553b172a4eccd34acdc9 Jul 15 08:07:24.032927 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:24.032838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" event={"ID":"a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a","Type":"ContainerStarted","Data":"889d26760e51c5fbb78555b3fcdf116e9605db235772553b172a4eccd34acdc9"} Jul 15 08:07:26.304062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:26.304029 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:26.304462 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:26.304091 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:26.308719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:26.308693 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:27.051337 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.051295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" event={"ID":"a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a","Type":"ContainerStarted","Data":"f7e3f44b16e042a70f449eafbed610255444fe7ee916ae7193bdc6976e087816"} Jul 15 08:07:27.051656 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.051625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:27.055174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.055151 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6df465f9dd-gxsk8" Jul 15 08:07:27.070301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.070256 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" podStartSLOduration=1.786911675 podStartE2EDuration="4.0702431s" totalImagesPullingTime="2.283331425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:27.068075626 +0000 UTC m=+410.459480249" watchObservedRunningTime="2026-07-15 08:07:27.0702431 +0000 UTC m=+410.461647724" Jul 15 08:07:27.119434 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.119403 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:07:27.476235 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.476200 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8qg54"] Jul 15 08:07:27.480237 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.480218 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:27.483545 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.483523 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nxv64\"" Jul 15 08:07:27.491940 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.491916 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8qg54"] Jul 15 08:07:27.541463 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.541428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt294\" (UniqueName: \"kubernetes.io/projected/12a196c1-1cad-43ee-8bc6-49088a1339bd-kube-api-access-rt294\") pod \"authorino-operator-75cc8d5cb7-8qg54\" (UID: \"12a196c1-1cad-43ee-8bc6-49088a1339bd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:27.642371 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.642276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt294\" (UniqueName: \"kubernetes.io/projected/12a196c1-1cad-43ee-8bc6-49088a1339bd-kube-api-access-rt294\") pod \"authorino-operator-75cc8d5cb7-8qg54\" (UID: \"12a196c1-1cad-43ee-8bc6-49088a1339bd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:27.661370 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.661324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt294\" (UniqueName: \"kubernetes.io/projected/12a196c1-1cad-43ee-8bc6-49088a1339bd-kube-api-access-rt294\") pod \"authorino-operator-75cc8d5cb7-8qg54\" (UID: \"12a196c1-1cad-43ee-8bc6-49088a1339bd\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:27.790059 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.789959 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:27.925700 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:27.925671 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8qg54"] Jul 15 08:07:27.929573 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:27.929542 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a196c1_1cad_43ee_8bc6_49088a1339bd.slice/crio-e95648e2104f043181f5d29640d0bfaacf5e428a50ecf679f95e1c5e4d6cbf5c WatchSource:0}: Error finding container e95648e2104f043181f5d29640d0bfaacf5e428a50ecf679f95e1c5e4d6cbf5c: Status 404 returned error can't find the container with id e95648e2104f043181f5d29640d0bfaacf5e428a50ecf679f95e1c5e4d6cbf5c Jul 15 08:07:28.057634 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:28.057533 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" event={"ID":"12a196c1-1cad-43ee-8bc6-49088a1339bd","Type":"ContainerStarted","Data":"e95648e2104f043181f5d29640d0bfaacf5e428a50ecf679f95e1c5e4d6cbf5c"} Jul 15 08:07:31.074255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:31.074224 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" event={"ID":"12a196c1-1cad-43ee-8bc6-49088a1339bd","Type":"ContainerStarted","Data":"682437394b669294986d7aad55e9185e1eab3eccc6bff0745151fda1314947c6"} Jul 15 08:07:31.074648 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:31.074296 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:31.097489 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:31.097439 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" podStartSLOduration=1.5588800539999998 podStartE2EDuration="4.097422212s" totalImagesPullingTime="2.538542158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:31.096898868 +0000 UTC m=+414.488303511" watchObservedRunningTime="2026-07-15 08:07:31.097422212 +0000 UTC m=+414.488826835" Jul 15 08:07:32.358746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.358701 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc"] Jul 15 08:07:32.361918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.361898 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.364767 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.364747 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fgfbk\"" Jul 15 08:07:32.380883 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.380854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.380990 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.380892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjsd4\" (UniqueName: \"kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.398236 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.398206 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc"] Jul 15 08:07:32.481787 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.481748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.481787 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.481794 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjsd4\" (UniqueName: \"kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.482180 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.482161 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.490698 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.490672 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjsd4\" (UniqueName: \"kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.671464 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.671381 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:32.810262 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:32.810228 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc"] Jul 15 08:07:32.813383 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:32.813341 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc67565_ce97_4e49_b12b_5687d3924bf0.slice/crio-58ef83cf6c05b8e57a72ad7f3fa26ad5f643097c24701190092c317cf52daabc WatchSource:0}: Error finding container 58ef83cf6c05b8e57a72ad7f3fa26ad5f643097c24701190092c317cf52daabc: Status 404 returned error can't find the container with id 58ef83cf6c05b8e57a72ad7f3fa26ad5f643097c24701190092c317cf52daabc Jul 15 08:07:33.084535 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:33.084503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" event={"ID":"4fc67565-ce97-4e49-b12b-5687d3924bf0","Type":"ContainerStarted","Data":"58ef83cf6c05b8e57a72ad7f3fa26ad5f643097c24701190092c317cf52daabc"} Jul 15 08:07:35.224349 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.224312 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt"] Jul 15 08:07:35.227825 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.227806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:35.235815 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.235797 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kqr78\"" Jul 15 08:07:35.258622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.258594 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt"] Jul 15 08:07:35.306608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.306572 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56r4r\" (UniqueName: \"kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r\") pod \"limitador-operator-controller-manager-859d7b78bd-lssjt\" (UID: \"8e4eb690-719b-4405-a2d2-59bc10efe789\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:35.407289 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.407256 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56r4r\" (UniqueName: \"kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r\") pod \"limitador-operator-controller-manager-859d7b78bd-lssjt\" (UID: \"8e4eb690-719b-4405-a2d2-59bc10efe789\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:35.421158 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.421129 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56r4r\" (UniqueName: \"kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r\") pod \"limitador-operator-controller-manager-859d7b78bd-lssjt\" (UID: \"8e4eb690-719b-4405-a2d2-59bc10efe789\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:35.538087 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.538005 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:35.687232 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:35.687209 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt"] Jul 15 08:07:35.689195 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:35.689167 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4eb690_719b_4405_a2d2_59bc10efe789.slice/crio-517b381fb799da50689994b5c4d6d9e36fa9ac457c47e488180cacf1cacf736b WatchSource:0}: Error finding container 517b381fb799da50689994b5c4d6d9e36fa9ac457c47e488180cacf1cacf736b: Status 404 returned error can't find the container with id 517b381fb799da50689994b5c4d6d9e36fa9ac457c47e488180cacf1cacf736b Jul 15 08:07:36.103940 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:36.103896 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" event={"ID":"8e4eb690-719b-4405-a2d2-59bc10efe789","Type":"ContainerStarted","Data":"517b381fb799da50689994b5c4d6d9e36fa9ac457c47e488180cacf1cacf736b"} Jul 15 08:07:38.060235 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:38.060202 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4cwxh" Jul 15 08:07:39.119381 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:39.119322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" event={"ID":"8e4eb690-719b-4405-a2d2-59bc10efe789","Type":"ContainerStarted","Data":"a0751f8531f6781bb28ff4e2a94c218d4750a3b4778717cb2bc3fd492e62292b"} Jul 15 08:07:39.119819 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:39.119404 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:39.145446 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:39.145403 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" podStartSLOduration=1.715895237 podStartE2EDuration="4.145384403s" totalImagesPullingTime="2.429489166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:39.144199107 +0000 UTC m=+422.535603751" watchObservedRunningTime="2026-07-15 08:07:39.145384403 +0000 UTC m=+422.536789025" Jul 15 08:07:41.131156 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:41.131124 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" event={"ID":"4fc67565-ce97-4e49-b12b-5687d3924bf0","Type":"ContainerStarted","Data":"f5cf3939d5fe27393939810f508f4ed6eadf531958827304eddc2e4106704197"} Jul 15 08:07:41.131598 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:41.131284 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:41.154029 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:41.153983 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" podStartSLOduration=1.425704066 podStartE2EDuration="9.15397108s" totalImagesPullingTime="7.728267014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:41.15156296 +0000 UTC m=+424.542967583" watchObservedRunningTime="2026-07-15 08:07:41.15397108 +0000 UTC m=+424.545375702" Jul 15 08:07:42.081098 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:42.081068 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8qg54" Jul 15 08:07:50.128201 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:50.128173 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:52.138239 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.138204 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:52.139512 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.139484 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56485c58df-fljdx" podUID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" containerName="console" containerID="cri-o://d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca" gracePeriod=15 Jul 15 08:07:52.389625 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.389571 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:07:52.457108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457076 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457122 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457160 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksktg\" (UniqueName: \"kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457195 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457223 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457251 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457540 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457348 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config\") pod \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\" (UID: \"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5\") " Jul 15 08:07:52.457685 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457662 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:07:52.457749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457687 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config" (OuterVolumeSpecName: "console-config") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:07:52.457749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457723 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:07:52.457957 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.457932 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:07:52.459392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.459352 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:07:52.459392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.459374 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:07:52.459537 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.459520 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg" (OuterVolumeSpecName: "kube-api-access-ksktg") pod "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" (UID: "4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5"). InnerVolumeSpecName "kube-api-access-ksktg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:07:52.558859 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558829 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-oauth-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.558859 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558854 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.558859 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558864 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-service-ca\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.559038 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558872 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksktg\" (UniqueName: \"kubernetes.io/projected/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-kube-api-access-ksktg\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.559038 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558881 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-console-config\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.559038 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558889 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-trusted-ca-bundle\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:52.559038 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:52.558898 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5-oauth-serving-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:53.188043 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.188011 2581 generic.go:356] "Generic (PLEG): container finished" podID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" containerID="d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca" exitCode=0 Jul 15 08:07:53.188401 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.188055 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56485c58df-fljdx" Jul 15 08:07:53.188401 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.188102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56485c58df-fljdx" event={"ID":"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5","Type":"ContainerDied","Data":"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca"} Jul 15 08:07:53.188401 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.188153 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56485c58df-fljdx" event={"ID":"4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5","Type":"ContainerDied","Data":"0ca022aa3c0fdad9891d3fd2ff78aeac9584327a2698f83e4089643719e2284f"} Jul 15 08:07:53.188401 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.188179 2581 scope.go:117] "RemoveContainer" containerID="d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca" Jul 15 08:07:53.197190 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.197173 2581 scope.go:117] "RemoveContainer" containerID="d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca" Jul 15 08:07:53.197467 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:53.197447 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca\": container with ID starting with d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca not found: ID does not exist" containerID="d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca" Jul 15 08:07:53.197537 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.197478 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca"} err="failed to get container status \"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca\": rpc error: code = NotFound desc = could not find container \"d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca\": container with ID starting with d6efabeeea1588629c970bde33b2be67e7f93bf3b6321b7210a2f90c9c842dca not found: ID does not exist" Jul 15 08:07:53.226192 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.226170 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:07:53.231717 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.231700 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56485c58df-fljdx"] Jul 15 08:07:53.351918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:53.351882 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" path="/var/lib/kubelet/pods/4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5/volumes" Jul 15 08:07:54.036445 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.036413 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc"] Jul 15 08:07:54.036720 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.036680 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" containerName="manager" containerID="cri-o://f5cf3939d5fe27393939810f508f4ed6eadf531958827304eddc2e4106704197" gracePeriod=2 Jul 15 08:07:54.039117 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.039062 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.041299 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.041270 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc"] Jul 15 08:07:54.057182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.057158 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt"] Jul 15 08:07:54.057535 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.057416 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" containerName="manager" containerID="cri-o://a0751f8531f6781bb28ff4e2a94c218d4750a3b4778717cb2bc3fd492e62292b" gracePeriod=2 Jul 15 08:07:54.061499 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.061470 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.064351 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.064329 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt"] Jul 15 08:07:54.070329 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.070296 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:07:54.071534 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.070929 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" containerName="manager" Jul 15 08:07:54.071676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071653 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" containerName="manager" Jul 15 08:07:54.071778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071695 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" containerName="console" Jul 15 08:07:54.071778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071705 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" containerName="console" Jul 15 08:07:54.071778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071719 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" containerName="manager" Jul 15 08:07:54.071778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071727 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" containerName="manager" Jul 15 08:07:54.071984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071876 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" containerName="manager" Jul 15 08:07:54.071984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071890 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c210ea9-b4b1-4291-a5b6-5b7b10a6bab5" containerName="console" Jul 15 08:07:54.071984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.071899 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" containerName="manager" Jul 15 08:07:54.075033 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.075014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.077505 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.077479 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:54.079915 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.079892 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.088665 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.088643 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz"] Jul 15 08:07:54.091622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.091606 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:54.095843 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.095821 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:07:54.109583 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.109554 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz"] Jul 15 08:07:54.115107 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.115058 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.136258 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.136233 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:54.179464 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.179437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.179566 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.179490 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652cw\" (UniqueName: \"kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.179566 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.179549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfkf\" (UniqueName: \"kubernetes.io/projected/7e440794-5f15-40bf-9288-712c16bd41fa-kube-api-access-dbfkf\") pod \"limitador-operator-controller-manager-859d7b78bd-kmbwz\" (UID: \"7e440794-5f15-40bf-9288-712c16bd41fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:54.199534 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.199500 2581 generic.go:356] "Generic (PLEG): container finished" podID="8e4eb690-719b-4405-a2d2-59bc10efe789" containerID="a0751f8531f6781bb28ff4e2a94c218d4750a3b4778717cb2bc3fd492e62292b" exitCode=0 Jul 15 08:07:54.202722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.202694 2581 generic.go:356] "Generic (PLEG): container finished" podID="4fc67565-ce97-4e49-b12b-5687d3924bf0" containerID="f5cf3939d5fe27393939810f508f4ed6eadf531958827304eddc2e4106704197" exitCode=0 Jul 15 08:07:54.280849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.280805 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfkf\" (UniqueName: \"kubernetes.io/projected/7e440794-5f15-40bf-9288-712c16bd41fa-kube-api-access-dbfkf\") pod \"limitador-operator-controller-manager-859d7b78bd-kmbwz\" (UID: \"7e440794-5f15-40bf-9288-712c16bd41fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:54.280995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.280899 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.280995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.280942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652cw\" (UniqueName: \"kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.281398 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.281375 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.293409 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.293385 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-652cw\" (UniqueName: \"kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mw9s\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.296790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.296761 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfkf\" (UniqueName: \"kubernetes.io/projected/7e440794-5f15-40bf-9288-712c16bd41fa-kube-api-access-dbfkf\") pod \"limitador-operator-controller-manager-859d7b78bd-kmbwz\" (UID: \"7e440794-5f15-40bf-9288-712c16bd41fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:54.316089 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.316070 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.318047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.318030 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:54.318669 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.318641 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:54.320689 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.320666 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.322787 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.322767 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:54.325402 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:54.325383 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:54.382073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.382049 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume\") pod \"4fc67565-ce97-4e49-b12b-5687d3924bf0\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " Jul 15 08:07:54.382174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.382106 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56r4r\" (UniqueName: \"kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r\") pod \"8e4eb690-719b-4405-a2d2-59bc10efe789\" (UID: \"8e4eb690-719b-4405-a2d2-59bc10efe789\") " Jul 15 08:07:54.382174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.382144 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjsd4\" (UniqueName: \"kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4\") pod \"4fc67565-ce97-4e49-b12b-5687d3924bf0\" (UID: \"4fc67565-ce97-4e49-b12b-5687d3924bf0\") " Jul 15 08:07:54.382624 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.382582 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4fc67565-ce97-4e49-b12b-5687d3924bf0" (UID: "4fc67565-ce97-4e49-b12b-5687d3924bf0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:07:54.384027 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.384009 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r" (OuterVolumeSpecName: "kube-api-access-56r4r") pod "8e4eb690-719b-4405-a2d2-59bc10efe789" (UID: "8e4eb690-719b-4405-a2d2-59bc10efe789"). InnerVolumeSpecName "kube-api-access-56r4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:07:54.384108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.384086 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4" (OuterVolumeSpecName: "kube-api-access-wjsd4") pod "4fc67565-ce97-4e49-b12b-5687d3924bf0" (UID: "4fc67565-ce97-4e49-b12b-5687d3924bf0"). InnerVolumeSpecName "kube-api-access-wjsd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:07:54.479143 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.479113 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:54.483295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.483276 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjsd4\" (UniqueName: \"kubernetes.io/projected/4fc67565-ce97-4e49-b12b-5687d3924bf0-kube-api-access-wjsd4\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:54.483407 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.483298 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fc67565-ce97-4e49-b12b-5687d3924bf0-extensions-socket-volume\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:54.483407 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.483313 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56r4r\" (UniqueName: \"kubernetes.io/projected/8e4eb690-719b-4405-a2d2-59bc10efe789-kube-api-access-56r4r\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:07:54.486810 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.486787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:54.635510 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.635487 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:07:54.638258 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:54.638224 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d6d9ca_1f27_49c7_a43e_fb6cffbdb42e.slice/crio-a495bc4b06530044c6aec0a99c4438410c136a1f70a70e9ce7ccd86111160399 WatchSource:0}: Error finding container a495bc4b06530044c6aec0a99c4438410c136a1f70a70e9ce7ccd86111160399: Status 404 returned error can't find the container with id a495bc4b06530044c6aec0a99c4438410c136a1f70a70e9ce7ccd86111160399 Jul 15 08:07:54.654200 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:54.654181 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz"] Jul 15 08:07:54.655718 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:07:54.655693 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e440794_5f15_40bf_9288_712c16bd41fa.slice/crio-c117e34b3d6c628c8e9cb4566900577543399dfa6fda92a23b13aae5fd524e0f WatchSource:0}: Error finding container c117e34b3d6c628c8e9cb4566900577543399dfa6fda92a23b13aae5fd524e0f: Status 404 returned error can't find the container with id c117e34b3d6c628c8e9cb4566900577543399dfa6fda92a23b13aae5fd524e0f Jul 15 08:07:55.208295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.208269 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:55.208295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.208290 2581 scope.go:117] "RemoveContainer" containerID="f5cf3939d5fe27393939810f508f4ed6eadf531958827304eddc2e4106704197" Jul 15 08:07:55.210072 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.210045 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" event={"ID":"7e440794-5f15-40bf-9288-712c16bd41fa","Type":"ContainerStarted","Data":"e7251cd1e26266fb4ff59f9340e5342714bf6eddcb3e8aae237b83685528629f"} Jul 15 08:07:55.210182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.210093 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" event={"ID":"7e440794-5f15-40bf-9288-712c16bd41fa","Type":"ContainerStarted","Data":"c117e34b3d6c628c8e9cb4566900577543399dfa6fda92a23b13aae5fd524e0f"} Jul 15 08:07:55.210182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.210137 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:07:55.211930 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.211906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" event={"ID":"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e","Type":"ContainerStarted","Data":"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526"} Jul 15 08:07:55.212056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.211963 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" event={"ID":"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e","Type":"ContainerStarted","Data":"a495bc4b06530044c6aec0a99c4438410c136a1f70a70e9ce7ccd86111160399"} Jul 15 08:07:55.212056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.211996 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:07:55.213312 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.213289 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:55.217650 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.217634 2581 scope.go:117] "RemoveContainer" containerID="a0751f8531f6781bb28ff4e2a94c218d4750a3b4778717cb2bc3fd492e62292b" Jul 15 08:07:55.219594 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:55.219572 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:55.225880 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:55.225857 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:07:55.227990 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:55.227966 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-d6flc\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-d6flc" Jul 15 08:07:55.294505 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.294465 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" podStartSLOduration=1.294450305 podStartE2EDuration="1.294450305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:55.294291956 +0000 UTC m=+438.685696600" watchObservedRunningTime="2026-07-15 08:07:55.294450305 +0000 UTC m=+438.685854930" Jul 15 08:07:55.351590 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.351555 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fc67565-ce97-4e49-b12b-5687d3924bf0" path="/var/lib/kubelet/pods/4fc67565-ce97-4e49-b12b-5687d3924bf0/volumes" Jul 15 08:07:55.351944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:07:55.351925 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" path="/var/lib/kubelet/pods/8e4eb690-719b-4405-a2d2-59bc10efe789/volumes" Jul 15 08:07:55.360566 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:07:55.360537 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lssjt\" is forbidden: User \"system:node:ip-10-0-141-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-240.ec2.internal' and this object" podUID="8e4eb690-719b-4405-a2d2-59bc10efe789" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lssjt" Jul 15 08:08:06.220731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:06.220660 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" Jul 15 08:08:06.221142 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:06.220738 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:08:06.241988 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:06.241942 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kmbwz" podStartSLOduration=12.241927864 podStartE2EDuration="12.241927864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:07:55.348769335 +0000 UTC m=+438.740173958" watchObservedRunningTime="2026-07-15 08:08:06.241927864 +0000 UTC m=+449.633332487" Jul 15 08:08:11.301055 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.301021 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:08:11.301464 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.301222 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" podUID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" containerName="manager" containerID="cri-o://38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526" gracePeriod=10 Jul 15 08:08:11.544480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.544455 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:08:11.548423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.548403 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.549315 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.549301 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:08:11.561854 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.561833 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:08:11.723627 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.723600 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-652cw\" (UniqueName: \"kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw\") pod \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " Jul 15 08:08:11.723627 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.723632 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume\") pod \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\" (UID: \"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e\") " Jul 15 08:08:11.723858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.723800 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwhn8\" (UniqueName: \"kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.723858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.723854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.724107 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.724079 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" (UID: "d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:08:11.725633 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.725613 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw" (OuterVolumeSpecName: "kube-api-access-652cw") pod "d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" (UID: "d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e"). InnerVolumeSpecName "kube-api-access-652cw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:08:11.824860 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.824787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.824984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.824857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwhn8\" (UniqueName: \"kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.824984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.824904 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-652cw\" (UniqueName: \"kubernetes.io/projected/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-kube-api-access-652cw\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:08:11.824984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.824920 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e-extensions-socket-volume\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:08:11.825212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.825189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.837273 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.837252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwhn8\" (UniqueName: \"kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8\") pod \"kuadrant-operator-controller-manager-86878c769f-fdk7x\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.857780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.857757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:11.993699 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:11.993676 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:08:11.996036 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:08:11.995998 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ad16db_beca_4b0a_95bc_5da79a227df6.slice/crio-65d728eaef2632313d0d5c56d2a22c10185d1557d23ba549558c1b3d96f03f28 WatchSource:0}: Error finding container 65d728eaef2632313d0d5c56d2a22c10185d1557d23ba549558c1b3d96f03f28: Status 404 returned error can't find the container with id 65d728eaef2632313d0d5c56d2a22c10185d1557d23ba549558c1b3d96f03f28 Jul 15 08:08:12.308076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.308043 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" event={"ID":"a1ad16db-beca-4b0a-95bc-5da79a227df6","Type":"ContainerStarted","Data":"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948"} Jul 15 08:08:12.308496 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.308112 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:12.308496 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.308135 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" event={"ID":"a1ad16db-beca-4b0a-95bc-5da79a227df6","Type":"ContainerStarted","Data":"65d728eaef2632313d0d5c56d2a22c10185d1557d23ba549558c1b3d96f03f28"} Jul 15 08:08:12.309299 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.309277 2581 generic.go:356] "Generic (PLEG): container finished" podID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" containerID="38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526" exitCode=0 Jul 15 08:08:12.309474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.309314 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" Jul 15 08:08:12.309474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.309316 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" event={"ID":"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e","Type":"ContainerDied","Data":"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526"} Jul 15 08:08:12.309474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.309444 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s" event={"ID":"d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e","Type":"ContainerDied","Data":"a495bc4b06530044c6aec0a99c4438410c136a1f70a70e9ce7ccd86111160399"} Jul 15 08:08:12.309474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.309472 2581 scope.go:117] "RemoveContainer" containerID="38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526" Jul 15 08:08:12.319149 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.319131 2581 scope.go:117] "RemoveContainer" containerID="38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526" Jul 15 08:08:12.319411 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:08:12.319391 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526\": container with ID starting with 38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526 not found: ID does not exist" containerID="38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526" Jul 15 08:08:12.319471 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.319417 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526"} err="failed to get container status \"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526\": rpc error: code = NotFound desc = could not find container \"38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526\": container with ID starting with 38c49573815b9364c27f81e4564840eb0760e5333c17ce3ecfeaed523fbc2526 not found: ID does not exist" Jul 15 08:08:12.328839 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.328802 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" podStartSLOduration=1.3287900879999999 podStartE2EDuration="1.328790088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:08:12.327217747 +0000 UTC m=+455.718622371" watchObservedRunningTime="2026-07-15 08:08:12.328790088 +0000 UTC m=+455.720194711" Jul 15 08:08:12.345623 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.345602 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:08:12.348599 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:12.348577 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mw9s"] Jul 15 08:08:13.351035 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:13.350998 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" path="/var/lib/kubelet/pods/d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e/volumes" Jul 15 08:08:23.316631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:23.316601 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:08:27.725900 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.725869 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p"] Jul 15 08:08:27.726257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.726204 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" containerName="manager" Jul 15 08:08:27.726257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.726215 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" containerName="manager" Jul 15 08:08:27.726325 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.726267 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7d6d9ca-1f27-49c7-a43e-fb6cffbdb42e" containerName="manager" Jul 15 08:08:27.729888 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.729865 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.732888 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.732866 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gnbkz\"" Jul 15 08:08:27.743277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.743255 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p"] Jul 15 08:08:27.744609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqpsr\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-kube-api-access-dqpsr\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744885 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744885 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.744885 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.744878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845787 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqpsr\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-kube-api-access-dqpsr\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.845972 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845954 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.845980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.846219 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.846336 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.846414 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846654 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.846488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.846846 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.846817 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.848249 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.848227 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.848602 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.848585 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.853585 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.853565 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:27.854039 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:27.854018 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqpsr\" (UniqueName: \"kubernetes.io/projected/0cfe7135-0067-4a8d-81b0-3cd0817de42d-kube-api-access-dqpsr\") pod \"maas-default-gateway-openshift-default-56654dcf7-75t2p\" (UID: \"0cfe7135-0067-4a8d-81b0-3cd0817de42d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:28.040414 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.040312 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:28.175616 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.175598 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p"] Jul 15 08:08:28.177951 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:08:28.177922 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfe7135_0067_4a8d_81b0_3cd0817de42d.slice/crio-cdcd28d69ea960f94eaa29639e961f2a75cd090f46efdece3affb61c3f9949c7 WatchSource:0}: Error finding container cdcd28d69ea960f94eaa29639e961f2a75cd090f46efdece3affb61c3f9949c7: Status 404 returned error can't find the container with id cdcd28d69ea960f94eaa29639e961f2a75cd090f46efdece3affb61c3f9949c7 Jul 15 08:08:28.180053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.180013 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:08:28.180140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.180103 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:08:28.180194 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.180176 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:08:28.383475 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.383439 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" event={"ID":"0cfe7135-0067-4a8d-81b0-3cd0817de42d","Type":"ContainerStarted","Data":"839dec653bc778769baf71e804fc75fdff6c759fcb66cd18573180c276e7bfdd"} Jul 15 08:08:28.383646 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.383488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" event={"ID":"0cfe7135-0067-4a8d-81b0-3cd0817de42d","Type":"ContainerStarted","Data":"cdcd28d69ea960f94eaa29639e961f2a75cd090f46efdece3affb61c3f9949c7"} Jul 15 08:08:28.405629 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:28.405582 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" podStartSLOduration=1.4055662230000001 podStartE2EDuration="1.405566223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:08:28.403330572 +0000 UTC m=+471.794735196" watchObservedRunningTime="2026-07-15 08:08:28.405566223 +0000 UTC m=+471.796970846" Jul 15 08:08:29.040616 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:29.040579 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:29.045506 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:29.045476 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:29.390745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:29.390636 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:29.391894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:29.391873 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:08:43.922380 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:43.922321 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:08:43.927321 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:43.927299 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:08:43.930069 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:43.930052 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-grskd\"" Jul 15 08:08:43.933277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:43.933256 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:08:43.989100 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:43.989075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjw9x\" (UniqueName: \"kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x\") pod \"authorino-75d556bd77-tjk2b\" (UID: \"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783\") " pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:08:44.090056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:44.090030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjw9x\" (UniqueName: \"kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x\") pod \"authorino-75d556bd77-tjk2b\" (UID: \"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783\") " pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:08:44.098169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:44.098143 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjw9x\" (UniqueName: \"kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x\") pod \"authorino-75d556bd77-tjk2b\" (UID: \"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783\") " pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:08:44.237005 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:44.236942 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:08:44.363532 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:44.363501 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:08:44.369141 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:08:44.369108 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a5a7284_0e8c_4a56_92b6_3ea09c1a3783.slice/crio-73748a74718db6c796d79862bf9bce0a43a7ad8fd56374422078eef4d0a9011e WatchSource:0}: Error finding container 73748a74718db6c796d79862bf9bce0a43a7ad8fd56374422078eef4d0a9011e: Status 404 returned error can't find the container with id 73748a74718db6c796d79862bf9bce0a43a7ad8fd56374422078eef4d0a9011e Jul 15 08:08:44.452433 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:44.452396 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjk2b" event={"ID":"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783","Type":"ContainerStarted","Data":"73748a74718db6c796d79862bf9bce0a43a7ad8fd56374422078eef4d0a9011e"} Jul 15 08:08:48.480731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:48.480696 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjk2b" event={"ID":"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783","Type":"ContainerStarted","Data":"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa"} Jul 15 08:08:48.498678 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:08:48.498635 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tjk2b" podStartSLOduration=2.377397412 podStartE2EDuration="5.498620188s" totalImagesPullingTime="3.121222776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:08:48.49626682 +0000 UTC m=+491.887671441" watchObservedRunningTime="2026-07-15 08:08:48.498620188 +0000 UTC m=+491.890024788" Jul 15 08:09:16.541477 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.541442 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:09:16.541988 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.541687 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tjk2b" podUID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" containerName="authorino" containerID="cri-o://5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa" gracePeriod=30 Jul 15 08:09:16.791138 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.791115 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:09:16.857866 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.857834 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjw9x\" (UniqueName: \"kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x\") pod \"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783\" (UID: \"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783\") " Jul 15 08:09:16.859965 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.859938 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x" (OuterVolumeSpecName: "kube-api-access-wjw9x") pod "8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" (UID: "8a5a7284-0e8c-4a56-92b6-3ea09c1a3783"). InnerVolumeSpecName "kube-api-access-wjw9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:09:16.958980 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:16.958949 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjw9x\" (UniqueName: \"kubernetes.io/projected/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783-kube-api-access-wjw9x\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:09:17.610156 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.610075 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" containerID="5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa" exitCode=0 Jul 15 08:09:17.610156 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.610117 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tjk2b" Jul 15 08:09:17.610156 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.610137 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjk2b" event={"ID":"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783","Type":"ContainerDied","Data":"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa"} Jul 15 08:09:17.610679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.610167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tjk2b" event={"ID":"8a5a7284-0e8c-4a56-92b6-3ea09c1a3783","Type":"ContainerDied","Data":"73748a74718db6c796d79862bf9bce0a43a7ad8fd56374422078eef4d0a9011e"} Jul 15 08:09:17.610679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.610183 2581 scope.go:117] "RemoveContainer" containerID="5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa" Jul 15 08:09:17.618774 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.618758 2581 scope.go:117] "RemoveContainer" containerID="5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa" Jul 15 08:09:17.619049 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:09:17.619025 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa\": container with ID starting with 5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa not found: ID does not exist" containerID="5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa" Jul 15 08:09:17.619153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.619054 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa"} err="failed to get container status \"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa\": rpc error: code = NotFound desc = could not find container \"5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa\": container with ID starting with 5015376b265603a000992fb6a183bbb3d0aef1ecc0acc65a553e3e13249d50aa not found: ID does not exist" Jul 15 08:09:17.630135 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.630115 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:09:17.631558 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:17.631538 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tjk2b"] Jul 15 08:09:19.351600 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:09:19.351567 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" path="/var/lib/kubelet/pods/8a5a7284-0e8c-4a56-92b6-3ea09c1a3783/volumes" Jul 15 08:10:06.832999 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.832963 2581 generic.go:356] "Generic (PLEG): container finished" podID="0cfe7135-0067-4a8d-81b0-3cd0817de42d" containerID="839dec653bc778769baf71e804fc75fdff6c759fcb66cd18573180c276e7bfdd" exitCode=0 Jul 15 08:10:06.833452 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.833022 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" event={"ID":"0cfe7135-0067-4a8d-81b0-3cd0817de42d","Type":"ContainerDied","Data":"839dec653bc778769baf71e804fc75fdff6c759fcb66cd18573180c276e7bfdd"} Jul 15 08:10:06.833452 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.833396 2581 scope.go:117] "RemoveContainer" containerID="839dec653bc778769baf71e804fc75fdff6c759fcb66cd18573180c276e7bfdd" Jul 15 08:10:06.834291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.834263 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:10:06.834341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.834332 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:10:06.834405 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:06.834384 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:10:07.843468 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:07.843433 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" event={"ID":"0cfe7135-0067-4a8d-81b0-3cd0817de42d","Type":"ContainerStarted","Data":"dd206ed52a04496fd03350c940c283d5b06e1c8fdbbe56426c22cfafe7256d72"} Jul 15 08:10:08.041332 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:08.041297 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:10:08.041561 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:08.041544 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-75t2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 08:10:08.041605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:08.041583 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" podUID="0cfe7135-0067-4a8d-81b0-3cd0817de42d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 08:10:09.041255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:09.041215 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-75t2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 08:10:09.041714 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:09.041279 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" podUID="0cfe7135-0067-4a8d-81b0-3cd0817de42d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 08:10:10.041125 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:10.041089 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-75t2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 08:10:10.041288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:10.041152 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" podUID="0cfe7135-0067-4a8d-81b0-3cd0817de42d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 08:10:11.041333 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:11.041294 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-75t2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 15 08:10:11.041743 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:11.041351 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" podUID="0cfe7135-0067-4a8d-81b0-3cd0817de42d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 15 08:10:12.057136 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:12.057105 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:10:12.057623 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:12.057386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:10:12.058098 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:12.058081 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-75t2p" Jul 15 08:10:37.083696 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:37.083665 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:10:37.084296 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:37.084179 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:10:45.318438 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.318404 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9"] Jul 15 08:10:45.318915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.318780 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" containerName="authorino" Jul 15 08:10:45.318915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.318793 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" containerName="authorino" Jul 15 08:10:45.318915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.318868 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5a7284-0e8c-4a56-92b6-3ea09c1a3783" containerName="authorino" Jul 15 08:10:45.321096 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.321077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.325573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.325547 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-schk4\"" Jul 15 08:10:45.325706 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.325589 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 08:10:45.325706 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.325606 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 15 08:10:45.325706 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.325547 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 08:10:45.333992 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.333970 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9"] Jul 15 08:10:45.425340 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.425544 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4l26\" (UniqueName: \"kubernetes.io/projected/70fa483a-76ba-4c08-8736-6ef8aa73b530-kube-api-access-d4l26\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.425544 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425429 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.425544 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.425544 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425507 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.425680 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.425559 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70fa483a-76ba-4c08-8736-6ef8aa73b530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527064 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527073 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4l26\" (UniqueName: \"kubernetes.io/projected/70fa483a-76ba-4c08-8736-6ef8aa73b530-kube-api-access-d4l26\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527148 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527442 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70fa483a-76ba-4c08-8736-6ef8aa73b530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527489 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527531 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.527643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.527578 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.529469 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.529444 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70fa483a-76ba-4c08-8736-6ef8aa73b530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.529751 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.529731 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70fa483a-76ba-4c08-8736-6ef8aa73b530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.535661 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.535636 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4l26\" (UniqueName: \"kubernetes.io/projected/70fa483a-76ba-4c08-8736-6ef8aa73b530-kube-api-access-d4l26\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9\" (UID: \"70fa483a-76ba-4c08-8736-6ef8aa73b530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.631586 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.631546 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:45.776774 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:45.776739 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9"] Jul 15 08:10:45.780098 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:10:45.780065 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fa483a_76ba_4c08_8736_6ef8aa73b530.slice/crio-683f2b030e15f177a1ec2aaba73f859b349066ed9a4f9cc60525576629f9ca24 WatchSource:0}: Error finding container 683f2b030e15f177a1ec2aaba73f859b349066ed9a4f9cc60525576629f9ca24: Status 404 returned error can't find the container with id 683f2b030e15f177a1ec2aaba73f859b349066ed9a4f9cc60525576629f9ca24 Jul 15 08:10:46.014982 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:46.014894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" event={"ID":"70fa483a-76ba-4c08-8736-6ef8aa73b530","Type":"ContainerStarted","Data":"683f2b030e15f177a1ec2aaba73f859b349066ed9a4f9cc60525576629f9ca24"} Jul 15 08:10:47.022024 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:47.021978 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" event={"ID":"70fa483a-76ba-4c08-8736-6ef8aa73b530","Type":"ContainerStarted","Data":"24ba6c64d01d2f9832372306290115fd4a35576f94d43fec5107a73a0c77daf0"} Jul 15 08:10:47.045303 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:47.045250 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" podStartSLOduration=0.923856182 podStartE2EDuration="2.04522973s" totalImagesPullingTime="1.121373548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:10:47.043444444 +0000 UTC m=+610.434849078" watchObservedRunningTime="2026-07-15 08:10:47.04522973 +0000 UTC m=+610.436634354" Jul 15 08:10:55.632108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:55.632074 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:55.632510 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:55.632148 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:55.644655 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:55.644625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:10:56.076529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:10:56.076447 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9" Jul 15 08:11:14.525262 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.525228 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns"] Jul 15 08:11:14.527841 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.527814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.530879 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.530856 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 08:11:14.544997 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.544967 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns"] Jul 15 08:11:14.560870 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.560833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.561060 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.560891 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.561060 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.560963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.561060 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.561001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nndr\" (UniqueName: \"kubernetes.io/projected/4ec3ad3c-e078-436a-a73f-06afc05a3a23-kube-api-access-7nndr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.561198 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.561060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.561198 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.561106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661794 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661834 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nndr\" (UniqueName: \"kubernetes.io/projected/4ec3ad3c-e078-436a-a73f-06afc05a3a23-kube-api-access-7nndr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.661989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.661946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.662313 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.662291 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.662397 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.662292 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.662397 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.662381 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.664174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.664151 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ec3ad3c-e078-436a-a73f-06afc05a3a23-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.664504 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.664480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ec3ad3c-e078-436a-a73f-06afc05a3a23-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.671238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.671211 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nndr\" (UniqueName: \"kubernetes.io/projected/4ec3ad3c-e078-436a-a73f-06afc05a3a23-kube-api-access-7nndr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns\" (UID: \"4ec3ad3c-e078-436a-a73f-06afc05a3a23\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.838345 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.838308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:14.974893 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.974859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns"] Jul 15 08:11:14.978865 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:11:14.978839 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec3ad3c_e078_436a_a73f_06afc05a3a23.slice/crio-d5a65de4c7ba9b4be85f65b0aefb2a6a2a17a7dbab5098bbcbcb34d3dcdde0b1 WatchSource:0}: Error finding container d5a65de4c7ba9b4be85f65b0aefb2a6a2a17a7dbab5098bbcbcb34d3dcdde0b1: Status 404 returned error can't find the container with id d5a65de4c7ba9b4be85f65b0aefb2a6a2a17a7dbab5098bbcbcb34d3dcdde0b1 Jul 15 08:11:14.980604 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:14.980582 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 08:11:15.159048 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:15.158957 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" event={"ID":"4ec3ad3c-e078-436a-a73f-06afc05a3a23","Type":"ContainerStarted","Data":"d5a65de4c7ba9b4be85f65b0aefb2a6a2a17a7dbab5098bbcbcb34d3dcdde0b1"} Jul 15 08:11:16.165438 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:16.165400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" event={"ID":"4ec3ad3c-e078-436a-a73f-06afc05a3a23","Type":"ContainerStarted","Data":"4f9bef3291b76d63a6e6118c451b71139de6e270803da89f417dc9988c731b40"} Jul 15 08:11:16.185441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:16.185386 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" podStartSLOduration=1.9712785990000001 podStartE2EDuration="2.18534837s" totalImagesPullingTime="214.069771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:11:16.184582363 +0000 UTC m=+639.575987007" watchObservedRunningTime="2026-07-15 08:11:16.18534837 +0000 UTC m=+639.576752994" Jul 15 08:11:24.838632 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:24.838578 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:24.839061 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:24.838653 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:24.851347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:24.851315 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:11:25.221701 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:11:25.221620 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns" Jul 15 08:12:39.968577 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:39.968484 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6db69d568c-mspsf"] Jul 15 08:12:39.971962 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:39.971938 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:39.975105 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:39.975076 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-grskd\"" Jul 15 08:12:39.976068 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:39.976050 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 08:12:39.981245 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:39.981221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db69d568c-mspsf"] Jul 15 08:12:40.121724 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.121686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95967b37-180e-41f7-bc94-de3cf05bae03-tls-cert\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.121915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.121742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt25j\" (UniqueName: \"kubernetes.io/projected/95967b37-180e-41f7-bc94-de3cf05bae03-kube-api-access-xt25j\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.223239 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.223151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95967b37-180e-41f7-bc94-de3cf05bae03-tls-cert\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.223239 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.223213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt25j\" (UniqueName: \"kubernetes.io/projected/95967b37-180e-41f7-bc94-de3cf05bae03-kube-api-access-xt25j\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.225669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.225642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95967b37-180e-41f7-bc94-de3cf05bae03-tls-cert\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.244687 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.244647 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt25j\" (UniqueName: \"kubernetes.io/projected/95967b37-180e-41f7-bc94-de3cf05bae03-kube-api-access-xt25j\") pod \"authorino-6db69d568c-mspsf\" (UID: \"95967b37-180e-41f7-bc94-de3cf05bae03\") " pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.282326 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.282286 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6db69d568c-mspsf" Jul 15 08:12:40.434643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.434613 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6db69d568c-mspsf"] Jul 15 08:12:40.437873 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:12:40.437840 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95967b37_180e_41f7_bc94_de3cf05bae03.slice/crio-db22dd801c8549be3f191c72adde1acafb3c246fff7b9186bfdb5a373daab39c WatchSource:0}: Error finding container db22dd801c8549be3f191c72adde1acafb3c246fff7b9186bfdb5a373daab39c: Status 404 returned error can't find the container with id db22dd801c8549be3f191c72adde1acafb3c246fff7b9186bfdb5a373daab39c Jul 15 08:12:40.559790 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:40.559700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db69d568c-mspsf" event={"ID":"95967b37-180e-41f7-bc94-de3cf05bae03","Type":"ContainerStarted","Data":"db22dd801c8549be3f191c72adde1acafb3c246fff7b9186bfdb5a373daab39c"} Jul 15 08:12:41.567615 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:41.567383 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6db69d568c-mspsf" event={"ID":"95967b37-180e-41f7-bc94-de3cf05bae03","Type":"ContainerStarted","Data":"716597bbd3562631647e9988bdc7f506f9a4be3659bb34b18cd8e6540e322e18"} Jul 15 08:12:41.595347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:12:41.595293 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6db69d568c-mspsf" podStartSLOduration=2.161678009 podStartE2EDuration="2.595273039s" totalImagesPullingTime="433.59503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:12:41.593622689 +0000 UTC m=+724.985027312" watchObservedRunningTime="2026-07-15 08:12:41.595273039 +0000 UTC m=+724.986677661" Jul 15 08:15:37.110582 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:15:37.110502 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:15:37.113144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:15:37.110796 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:18:46.632822 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:46.632736 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:18:46.633343 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:46.632979 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" podUID="a1ad16db-beca-4b0a-95bc-5da79a227df6" containerName="manager" containerID="cri-o://5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948" gracePeriod=10 Jul 15 08:18:50.025514 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.025483 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:18:50.068587 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.067702 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwhn8\" (UniqueName: \"kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8\") pod \"a1ad16db-beca-4b0a-95bc-5da79a227df6\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " Jul 15 08:18:50.068587 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.067778 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume\") pod \"a1ad16db-beca-4b0a-95bc-5da79a227df6\" (UID: \"a1ad16db-beca-4b0a-95bc-5da79a227df6\") " Jul 15 08:18:50.070290 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.070242 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a1ad16db-beca-4b0a-95bc-5da79a227df6" (UID: "a1ad16db-beca-4b0a-95bc-5da79a227df6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:18:50.076856 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.076817 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8" (OuterVolumeSpecName: "kube-api-access-bwhn8") pod "a1ad16db-beca-4b0a-95bc-5da79a227df6" (UID: "a1ad16db-beca-4b0a-95bc-5da79a227df6"). InnerVolumeSpecName "kube-api-access-bwhn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:18:50.168913 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.168787 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwhn8\" (UniqueName: \"kubernetes.io/projected/a1ad16db-beca-4b0a-95bc-5da79a227df6-kube-api-access-bwhn8\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:18:50.168913 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.168825 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad16db-beca-4b0a-95bc-5da79a227df6-extensions-socket-volume\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:18:50.366770 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.366727 2581 generic.go:356] "Generic (PLEG): container finished" podID="a1ad16db-beca-4b0a-95bc-5da79a227df6" containerID="5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948" exitCode=0 Jul 15 08:18:50.366965 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.366881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" event={"ID":"a1ad16db-beca-4b0a-95bc-5da79a227df6","Type":"ContainerDied","Data":"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948"} Jul 15 08:18:50.366965 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.366933 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" event={"ID":"a1ad16db-beca-4b0a-95bc-5da79a227df6","Type":"ContainerDied","Data":"65d728eaef2632313d0d5c56d2a22c10185d1557d23ba549558c1b3d96f03f28"} Jul 15 08:18:50.367095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.366967 2581 scope.go:117] "RemoveContainer" containerID="5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948" Jul 15 08:18:50.367159 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.367143 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x" Jul 15 08:18:50.384739 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.384721 2581 scope.go:117] "RemoveContainer" containerID="5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948" Jul 15 08:18:50.385059 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:18:50.385038 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948\": container with ID starting with 5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948 not found: ID does not exist" containerID="5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948" Jul 15 08:18:50.385152 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.385064 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948"} err="failed to get container status \"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948\": rpc error: code = NotFound desc = could not find container \"5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948\": container with ID starting with 5a0ad60fd0626897ce2f50c0d769a549d11fdc1868435d9020665b0eaac9e948 not found: ID does not exist" Jul 15 08:18:50.395603 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.395579 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:18:50.399202 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:50.399181 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fdk7x"] Jul 15 08:18:51.358116 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:18:51.358066 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ad16db-beca-4b0a-95bc-5da79a227df6" path="/var/lib/kubelet/pods/a1ad16db-beca-4b0a-95bc-5da79a227df6/volumes" Jul 15 08:19:47.882152 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.882110 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn"] Jul 15 08:19:47.882732 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.882680 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1ad16db-beca-4b0a-95bc-5da79a227df6" containerName="manager" Jul 15 08:19:47.882732 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.882701 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ad16db-beca-4b0a-95bc-5da79a227df6" containerName="manager" Jul 15 08:19:47.882848 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.882812 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1ad16db-beca-4b0a-95bc-5da79a227df6" containerName="manager" Jul 15 08:19:47.887029 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.887008 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:47.889995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.889975 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fgfbk\"" Jul 15 08:19:47.898392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.898373 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn"] Jul 15 08:19:47.935423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.935399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c87db7b-48e7-4b21-947d-ba13a4b05dba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:47.935538 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:47.935455 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5c9t\" (UniqueName: \"kubernetes.io/projected/6c87db7b-48e7-4b21-947d-ba13a4b05dba-kube-api-access-b5c9t\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.035841 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.035815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c87db7b-48e7-4b21-947d-ba13a4b05dba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.035940 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.035866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5c9t\" (UniqueName: \"kubernetes.io/projected/6c87db7b-48e7-4b21-947d-ba13a4b05dba-kube-api-access-b5c9t\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.036157 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.036138 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c87db7b-48e7-4b21-947d-ba13a4b05dba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.045485 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.045462 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5c9t\" (UniqueName: \"kubernetes.io/projected/6c87db7b-48e7-4b21-947d-ba13a4b05dba-kube-api-access-b5c9t\") pod \"kuadrant-operator-controller-manager-86878c769f-kwtqn\" (UID: \"6c87db7b-48e7-4b21-947d-ba13a4b05dba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.197133 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.197075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.335135 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.335066 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn"] Jul 15 08:19:48.338067 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:19:48.338027 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c87db7b_48e7_4b21_947d_ba13a4b05dba.slice/crio-37096911bd7607c70fae074e770048c02a18e57e5c3569f6c92e1e7201804b83 WatchSource:0}: Error finding container 37096911bd7607c70fae074e770048c02a18e57e5c3569f6c92e1e7201804b83: Status 404 returned error can't find the container with id 37096911bd7607c70fae074e770048c02a18e57e5c3569f6c92e1e7201804b83 Jul 15 08:19:48.630445 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.630405 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" event={"ID":"6c87db7b-48e7-4b21-947d-ba13a4b05dba","Type":"ContainerStarted","Data":"2a14733af60af1ad9716b2a330a2ef84662b05a633345807d1c7d64a9025dcd0"} Jul 15 08:19:48.630634 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.630469 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:19:48.630634 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.630492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" event={"ID":"6c87db7b-48e7-4b21-947d-ba13a4b05dba","Type":"ContainerStarted","Data":"37096911bd7607c70fae074e770048c02a18e57e5c3569f6c92e1e7201804b83"} Jul 15 08:19:48.650833 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:48.650790 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" podStartSLOduration=1.650771531 podStartE2EDuration="1.650771531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:19:48.650469478 +0000 UTC m=+1152.041874101" watchObservedRunningTime="2026-07-15 08:19:48.650771531 +0000 UTC m=+1152.042176155" Jul 15 08:19:59.637472 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:19:59.637393 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwtqn" Jul 15 08:20:37.136461 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:20:37.136335 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:20:37.140334 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:20:37.138487 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:25:37.161801 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:37.161694 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:25:37.166133 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:37.166106 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:25:49.601645 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.601614 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:25:49.606664 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.606637 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.610008 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.609978 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f0ce8ae8-openshift-default-dockercfg-rdhg6\"" Jul 15 08:25:49.627253 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.627232 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:25:49.707284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707252 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707503 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707503 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707503 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707573 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.707609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.707588 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-292p7\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.808908 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.808878 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.808908 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.808913 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.808930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-292p7\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.808956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809232 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809387 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809387 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809379 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809497 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809497 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809601 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809493 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809601 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809571 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.809711 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.809689 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.811702 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.811672 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.811831 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.811814 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.817797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.817728 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-292p7\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.817797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.817776 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token\") pod \"e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:49.917905 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:49.917876 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:50.062085 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.062061 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:25:50.064071 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:25:50.064044 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa62eca_bbfa_420d_bb65_3fac2e606d68.slice/crio-4285516da88d16be80b90569f146e73dd1c0c2068ae23d04fad4ae30524d8016 WatchSource:0}: Error finding container 4285516da88d16be80b90569f146e73dd1c0c2068ae23d04fad4ae30524d8016: Status 404 returned error can't find the container with id 4285516da88d16be80b90569f146e73dd1c0c2068ae23d04fad4ae30524d8016 Jul 15 08:25:50.066154 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.066118 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:25:50.066230 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.066189 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:25:50.066230 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.066225 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:25:50.374510 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.374469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerStarted","Data":"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938"} Jul 15 08:25:50.374704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.374538 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerStarted","Data":"4285516da88d16be80b90569f146e73dd1c0c2068ae23d04fad4ae30524d8016"} Jul 15 08:25:50.395850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.395800 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podStartSLOduration=1.395785196 podStartE2EDuration="1.395785196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:25:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:25:50.394545778 +0000 UTC m=+1513.785950444" watchObservedRunningTime="2026-07-15 08:25:50.395785196 +0000 UTC m=+1513.787189818" Jul 15 08:25:50.918506 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:50.918473 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:51.923151 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:51.923114 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:52.385605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:52.385571 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:25:52.386433 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:25:52.386413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:26:07.387749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.387717 2581 patch_prober.go:28] interesting pod/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 15 08:26:07.388126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.387783 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 15 08:26:07.462933 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.462902 2581 generic.go:356] "Generic (PLEG): container finished" podID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerID="58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938" exitCode=0 Jul 15 08:26:07.463076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.462976 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerDied","Data":"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938"} Jul 15 08:26:07.463400 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.463386 2581 scope.go:117] "RemoveContainer" containerID="58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938" Jul 15 08:26:07.464432 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.464398 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:26:07.464541 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.464468 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:26:07.464541 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:07.464499 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:26:08.471311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:08.471265 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerStarted","Data":"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536"} Jul 15 08:26:08.919024 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:08.918990 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:26:08.919260 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:08.919240 2581 patch_prober.go:28] interesting pod/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 15 08:26:08.919308 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:08.919284 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 15 08:26:09.919323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:09.919285 2581 patch_prober.go:28] interesting pod/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 15 08:26:09.919737 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:09.919343 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 15 08:26:10.919053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:10.919019 2581 patch_prober.go:28] interesting pod/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 15 08:26:10.919227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:10.919069 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 15 08:26:11.919103 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:11.919068 2581 patch_prober.go:28] interesting pod/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 15 08:26:11.919537 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:11.919127 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 15 08:26:12.922347 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:12.922319 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:26:12.922807 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:12.922585 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:26:12.923350 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:26:12.923330 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:27:21.232287 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.232251 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:27:21.233235 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.232550 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" containerID="cri-o://fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536" gracePeriod=30 Jul 15 08:27:21.795574 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.795544 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:27:21.797950 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.797934 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.801006 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.800987 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-cdca1605-gw-openshift-default-dockercfg-p4xg7\"" Jul 15 08:27:21.813565 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.813545 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:27:21.904043 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904011 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904184 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904184 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904081 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904184 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904112 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904184 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904184 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904394 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904202 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904394 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrl8s\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:21.904394 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:21.904261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005156 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005316 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005316 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005474 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005431 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrl8s\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005665 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005665 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005775 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005775 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005726 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005876 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005837 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.005966 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.005942 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.006104 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.006086 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.007677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.007652 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.007766 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.007664 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.013475 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.013451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrl8s\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.013715 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.013698 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token\") pod \"e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.108918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.108886 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:22.248182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.248156 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:27:22.250007 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:27:22.249977 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293d95e4_fa89_4a4e_ba7e_3cac9da1584c.slice/crio-877b8d5198395266fdcbf67e106252d7ecf0ae53ff1cb959273126a9602db714 WatchSource:0}: Error finding container 877b8d5198395266fdcbf67e106252d7ecf0ae53ff1cb959273126a9602db714: Status 404 returned error can't find the container with id 877b8d5198395266fdcbf67e106252d7ecf0ae53ff1cb959273126a9602db714 Jul 15 08:27:22.252193 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.252167 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:22.252293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.252225 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:22.252293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.252256 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:22.836884 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.836844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerStarted","Data":"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073"} Jul 15 08:27:22.837073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.836906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerStarted","Data":"877b8d5198395266fdcbf67e106252d7ecf0ae53ff1cb959273126a9602db714"} Jul 15 08:27:22.860487 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:22.860443 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podStartSLOduration=1.860415859 podStartE2EDuration="1.860415859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:27:22.85984731 +0000 UTC m=+1606.251251956" watchObservedRunningTime="2026-07-15 08:27:22.860415859 +0000 UTC m=+1606.251820481" Jul 15 08:27:23.109850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:23.109770 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:23.114162 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:23.114139 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:23.841679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:23.841651 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:23.842473 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:23.842452 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:26.483752 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.483729 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:27:26.645182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645149 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645182 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645200 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645264 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645290 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645333 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645391 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645419 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-292p7\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645498 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:27:26.645631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645524 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token\") pod \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\" (UID: \"9fa62eca-bbfa-420d-bb65-3fac2e606d68\") " Jul 15 08:27:26.645631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645574 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:27:26.645832 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645695 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:27:26.645832 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645694 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:27:26.645832 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645816 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.645994 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645835 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.645994 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645854 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.645994 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.645871 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.646097 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.646027 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:27:26.648017 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.647995 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:27:26.648108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.648064 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7" (OuterVolumeSpecName: "kube-api-access-292p7") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "kube-api-access-292p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:27:26.648153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.648108 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:27:26.648187 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.648143 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fa62eca-bbfa-420d-bb65-3fac2e606d68" (UID: "9fa62eca-bbfa-420d-bb65-3fac2e606d68"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:27:26.746863 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.746831 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.746863 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.746858 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.746863 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.746867 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.747081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.746876 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa62eca-bbfa-420d-bb65-3fac2e606d68-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.747081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.746885 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-292p7\" (UniqueName: \"kubernetes.io/projected/9fa62eca-bbfa-420d-bb65-3fac2e606d68-kube-api-access-292p7\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:27:26.863528 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.863498 2581 generic.go:356] "Generic (PLEG): container finished" podID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerID="fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536" exitCode=0 Jul 15 08:27:26.863693 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.863552 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" Jul 15 08:27:26.863693 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.863580 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerDied","Data":"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536"} Jul 15 08:27:26.863693 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.863632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp" event={"ID":"9fa62eca-bbfa-420d-bb65-3fac2e606d68","Type":"ContainerDied","Data":"4285516da88d16be80b90569f146e73dd1c0c2068ae23d04fad4ae30524d8016"} Jul 15 08:27:26.863693 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.863655 2581 scope.go:117] "RemoveContainer" containerID="fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536" Jul 15 08:27:26.873896 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.873878 2581 scope.go:117] "RemoveContainer" containerID="58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938" Jul 15 08:27:26.882909 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.882891 2581 scope.go:117] "RemoveContainer" containerID="fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536" Jul 15 08:27:26.883134 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:27:26.883119 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536\": container with ID starting with fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536 not found: ID does not exist" containerID="fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536" Jul 15 08:27:26.883181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.883140 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536"} err="failed to get container status \"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536\": rpc error: code = NotFound desc = could not find container \"fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536\": container with ID starting with fcac80dec11e7c2dde5fa560d77789d72453b9214b4ed98ad1a6a441c1582536 not found: ID does not exist" Jul 15 08:27:26.883181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.883155 2581 scope.go:117] "RemoveContainer" containerID="58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938" Jul 15 08:27:26.883346 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:27:26.883326 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938\": container with ID starting with 58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938 not found: ID does not exist" containerID="58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938" Jul 15 08:27:26.883581 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.883370 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938"} err="failed to get container status \"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938\": rpc error: code = NotFound desc = could not find container \"58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938\": container with ID starting with 58e931c6a83eba29a37d7cc4af75e71606fbe317362bc5e11c6a1f5fdabdc938 not found: ID does not exist" Jul 15 08:27:26.889547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.889521 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:27:26.895462 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:26.895443 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f0ce8ae8-openshift-default-74c569bb55-jh2lp"] Jul 15 08:27:27.352176 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:27.352148 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" path="/var/lib/kubelet/pods/9fa62eca-bbfa-420d-bb65-3fac2e606d68/volumes" Jul 15 08:27:39.291153 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.291101 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:51942->10.133.0.45:15021: read: connection reset by peer" start-of-body= Jul 15 08:27:39.291508 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.291177 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:51942->10.133.0.45:15021: read: connection reset by peer" Jul 15 08:27:39.932918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.932886 2581 generic.go:356] "Generic (PLEG): container finished" podID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerID="ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073" exitCode=0 Jul 15 08:27:39.933088 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.932938 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerDied","Data":"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073"} Jul 15 08:27:39.933307 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.933292 2581 scope.go:117] "RemoveContainer" containerID="ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073" Jul 15 08:27:39.934290 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.934258 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:39.934423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.934323 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:39.934423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:39.934370 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:27:40.939368 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:40.939333 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerStarted","Data":"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379"} Jul 15 08:27:41.109828 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:41.109784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:41.110053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:41.110035 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 08:27:41.110103 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:41.110083 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 08:27:42.109921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:42.109885 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 08:27:42.110301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:42.109939 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 08:27:43.109656 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:43.109618 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 08:27:43.109831 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:43.109678 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 08:27:44.109805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:44.109776 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 15 08:27:44.110164 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:44.109829 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 15 08:27:45.112859 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:45.112830 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:45.113234 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:45.113175 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:27:45.113789 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:27:45.113774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:28:27.766077 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:27.766040 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:28:27.766693 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:27.766241 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" containerID="cri-o://626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379" gracePeriod=30 Jul 15 08:28:30.115885 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:30.115846 2581 patch_prober.go:28] interesting pod/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:28:30.116249 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:30.115931 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:28:33.023294 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.023267 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:28:33.125406 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125374 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125545 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125415 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125545 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125439 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125545 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125470 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125707 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125552 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125707 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125586 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125707 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125614 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125707 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125694 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.125894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125708 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:28:33.125894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125728 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:28:33.125894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125735 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrl8s\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s\") pod \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\" (UID: \"293d95e4-fa89-4a4e-ba7e-3cac9da1584c\") " Jul 15 08:28:33.126047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125890 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:28:33.126047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.125923 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:28:33.126047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.126019 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.126047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.126038 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.126268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.126053 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.126268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.126066 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.126449 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.126387 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data" (OuterVolumeSpecName: "istio-data") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:28:33.127664 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.127640 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:28:33.127894 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.127873 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s" (OuterVolumeSpecName: "kube-api-access-lrl8s") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "kube-api-access-lrl8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:28:33.128145 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.128128 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token" (OuterVolumeSpecName: "istio-token") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:28:33.128145 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.128131 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "293d95e4-fa89-4a4e-ba7e-3cac9da1584c" (UID: "293d95e4-fa89-4a4e-ba7e-3cac9da1584c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:28:33.196432 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.196395 2581 generic.go:356] "Generic (PLEG): container finished" podID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerID="626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379" exitCode=0 Jul 15 08:28:33.196579 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.196435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerDied","Data":"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379"} Jul 15 08:28:33.196579 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.196464 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" Jul 15 08:28:33.196579 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.196494 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4" event={"ID":"293d95e4-fa89-4a4e-ba7e-3cac9da1584c","Type":"ContainerDied","Data":"877b8d5198395266fdcbf67e106252d7ecf0ae53ff1cb959273126a9602db714"} Jul 15 08:28:33.196579 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.196517 2581 scope.go:117] "RemoveContainer" containerID="626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379" Jul 15 08:28:33.205099 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.205082 2581 scope.go:117] "RemoveContainer" containerID="ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073" Jul 15 08:28:33.212811 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.212796 2581 scope.go:117] "RemoveContainer" containerID="626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379" Jul 15 08:28:33.213056 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:28:33.213035 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379\": container with ID starting with 626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379 not found: ID does not exist" containerID="626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379" Jul 15 08:28:33.213116 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.213064 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379"} err="failed to get container status \"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379\": rpc error: code = NotFound desc = could not find container \"626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379\": container with ID starting with 626938663ff3137782aa116ddea2c6638d5fca948d0b7dac488cd6ac3b31b379 not found: ID does not exist" Jul 15 08:28:33.213116 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.213080 2581 scope.go:117] "RemoveContainer" containerID="ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073" Jul 15 08:28:33.213308 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:28:33.213290 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073\": container with ID starting with ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073 not found: ID does not exist" containerID="ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073" Jul 15 08:28:33.213369 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.213314 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073"} err="failed to get container status \"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073\": rpc error: code = NotFound desc = could not find container \"ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073\": container with ID starting with ccdb843b8f20829173d89b49a441ecd8e4ab5cc1ded3b816d1645a8384755073 not found: ID does not exist" Jul 15 08:28:33.219257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.219235 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:28:33.223658 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.223638 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-cdca1605-gw-openshift-default-74855df5dc-bkjh4"] Jul 15 08:28:33.226864 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.226848 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.226921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.226867 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.226921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.226876 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrl8s\" (UniqueName: \"kubernetes.io/projected/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-kube-api-access-lrl8s\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.226921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.226885 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.226921 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.226894 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293d95e4-fa89-4a4e-ba7e-3cac9da1584c-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:28:33.351623 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:33.351599 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" path="/var/lib/kubelet/pods/293d95e4-fa89-4a4e-ba7e-3cac9da1584c/volumes" Jul 15 08:28:41.288454 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288425 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288761 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288772 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288787 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288793 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288801 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288806 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288813 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288818 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288873 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288880 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.288920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.288889 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa62eca-bbfa-420d-bb65-3fac2e606d68" containerName="istio-proxy" Jul 15 08:28:41.291666 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.291645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.294850 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.294831 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4dbdead5-openshift-default-dockercfg-qbxfk\"" Jul 15 08:28:41.303443 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.303421 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:28:41.400645 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.400821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400659 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.400821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400695 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqvb\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.400821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400720 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.400821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400745 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.401031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.401031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.401031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.401031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.400955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502052 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqvb\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502052 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502238 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502272 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502769 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.502939 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502790 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.503025 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.502985 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.503077 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.503017 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.503281 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.503264 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.505024 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.504997 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.505220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.505196 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.515430 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.515404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.515561 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.515544 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqvb\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb\") pod \"e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.602008 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.601978 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:41.743578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.743556 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:28:41.745904 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:28:41.745881 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf0ecfd_6678_484d_9bcb_a549baab0dc2.slice/crio-6a70b0f230909465573b345b70ee2e9b22b24f4659e0cdc63bbac58c5fb3c8ac WatchSource:0}: Error finding container 6a70b0f230909465573b345b70ee2e9b22b24f4659e0cdc63bbac58c5fb3c8ac: Status 404 returned error can't find the container with id 6a70b0f230909465573b345b70ee2e9b22b24f4659e0cdc63bbac58c5fb3c8ac Jul 15 08:28:41.748095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.748060 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:41.748174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.748130 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:41.748174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:41.748163 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:42.242582 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:42.242542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerStarted","Data":"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e"} Jul 15 08:28:42.242733 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:42.242600 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerStarted","Data":"6a70b0f230909465573b345b70ee2e9b22b24f4659e0cdc63bbac58c5fb3c8ac"} Jul 15 08:28:42.270777 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:42.270725 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podStartSLOduration=1.270704944 podStartE2EDuration="1.270704944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:28:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:28:42.270202345 +0000 UTC m=+1685.661606974" watchObservedRunningTime="2026-07-15 08:28:42.270704944 +0000 UTC m=+1685.662109550" Jul 15 08:28:42.602132 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:42.602079 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:42.606898 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:42.606874 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:43.247891 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:43.247863 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:43.248692 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:43.248672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:53.317761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.317729 2581 generic.go:356] "Generic (PLEG): container finished" podID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerID="8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e" exitCode=0 Jul 15 08:28:53.318159 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.317804 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerDied","Data":"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e"} Jul 15 08:28:53.318210 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.318173 2581 scope.go:117] "RemoveContainer" containerID="8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e" Jul 15 08:28:53.319132 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.319108 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:53.319185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.319177 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:53.319220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:53.319207 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:28:54.324182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:54.324142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerStarted","Data":"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab"} Jul 15 08:28:54.602394 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:54.602338 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:54.603455 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:54.603433 2581 patch_prober.go:28] interesting pod/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 08:28:54.603525 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:54.603482 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 08:28:55.603074 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:55.603043 2581 patch_prober.go:28] interesting pod/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 08:28:55.603472 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:55.603102 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 08:28:56.603164 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:56.603134 2581 patch_prober.go:28] interesting pod/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 08:28:56.603578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:56.603184 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 08:28:57.603291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:57.603256 2581 patch_prober.go:28] interesting pod/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 15 08:28:57.603794 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:57.603317 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 15 08:28:58.248085 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:58.248053 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:58.606007 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:58.605984 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:28:58.606923 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:28:58.606906 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:30:13.834239 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:13.834204 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:30:13.834758 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:13.834404 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" containerID="cri-o://eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab" gracePeriod=30 Jul 15 08:30:14.135892 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.135817 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:30:14.136267 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.136254 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="293d95e4-fa89-4a4e-ba7e-3cac9da1584c" containerName="istio-proxy" Jul 15 08:30:14.139132 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.139113 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.141605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.141582 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-93b7ed10-openshift-default-dockercfg-gwvg7\"" Jul 15 08:30:14.151851 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.151831 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:30:14.190715 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.190853 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7lg8\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.190853 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.190853 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190779 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.190853 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.190853 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.191031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.191031 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.190958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.191095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.191030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291777 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7lg8\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291887 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.291938 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292250 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.291966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292250 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292290 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292392 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292307 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292483 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292405 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292571 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292549 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.292738 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.292718 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.294023 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.294003 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.294233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.294215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.299924 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.299900 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.300127 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.300111 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7lg8\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8\") pod \"e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.449196 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.449121 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:14.581945 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.581916 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:30:14.583631 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:30:14.583602 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bff8ed_fa5c_44ca_9386_4a037149016b.slice/crio-47a58c39d9e25e9ac9517c20c211826411996b401f0442343ee99966fa25553f WatchSource:0}: Error finding container 47a58c39d9e25e9ac9517c20c211826411996b401f0442343ee99966fa25553f: Status 404 returned error can't find the container with id 47a58c39d9e25e9ac9517c20c211826411996b401f0442343ee99966fa25553f Jul 15 08:30:14.585839 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.585807 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:14.585944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.585882 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:14.585944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.585924 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:14.710282 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.710189 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerStarted","Data":"1fb5444d149bf884f322dfec32c33bfd89e267570c6fa8cc6b83f60c0e519ce3"} Jul 15 08:30:14.710282 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.710256 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerStarted","Data":"47a58c39d9e25e9ac9517c20c211826411996b401f0442343ee99966fa25553f"} Jul 15 08:30:14.730659 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:14.730608 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podStartSLOduration=0.730592362 podStartE2EDuration="730.592362ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:30:14.729010831 +0000 UTC m=+1778.120415476" watchObservedRunningTime="2026-07-15 08:30:14.730592362 +0000 UTC m=+1778.121996985" Jul 15 08:30:15.450348 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:15.450313 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:15.455749 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:15.455724 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:15.715465 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:15.715388 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:15.716412 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:15.716395 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:19.120183 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.120158 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:30:19.238373 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238330 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238401 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238428 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvqvb\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238447 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238494 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238519 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238514 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238544 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238580 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238610 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data\") pod \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\" (UID: \"daf0ecfd-6678-484d-9bcb-a549baab0dc2\") " Jul 15 08:30:19.238895 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238603 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:30:19.238895 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238826 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:30:19.239009 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.238870 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:30:19.239199 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.239174 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:30:19.239639 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.239613 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:30:19.240670 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.240648 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:30:19.240890 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.240872 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:30:19.241049 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.241033 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:30:19.241301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.241280 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb" (OuterVolumeSpecName: "kube-api-access-pvqvb") pod "daf0ecfd-6678-484d-9bcb-a549baab0dc2" (UID: "daf0ecfd-6678-484d-9bcb-a549baab0dc2"). InnerVolumeSpecName "kube-api-access-pvqvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:30:19.339676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339629 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339672 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvqvb\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-kube-api-access-pvqvb\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339676 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339685 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339697 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339709 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339719 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339729 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339738 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.339928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.339748 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf0ecfd-6678-484d-9bcb-a549baab0dc2-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:30:19.736127 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.736031 2581 generic.go:356] "Generic (PLEG): container finished" podID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerID="eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab" exitCode=0 Jul 15 08:30:19.736127 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.736098 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" Jul 15 08:30:19.736127 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.736112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerDied","Data":"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab"} Jul 15 08:30:19.736375 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.736171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w" event={"ID":"daf0ecfd-6678-484d-9bcb-a549baab0dc2","Type":"ContainerDied","Data":"6a70b0f230909465573b345b70ee2e9b22b24f4659e0cdc63bbac58c5fb3c8ac"} Jul 15 08:30:19.736375 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.736194 2581 scope.go:117] "RemoveContainer" containerID="eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab" Jul 15 08:30:19.744616 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.744598 2581 scope.go:117] "RemoveContainer" containerID="8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e" Jul 15 08:30:19.752140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.752123 2581 scope.go:117] "RemoveContainer" containerID="eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab" Jul 15 08:30:19.752511 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:30:19.752470 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab\": container with ID starting with eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab not found: ID does not exist" containerID="eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab" Jul 15 08:30:19.752622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.752511 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab"} err="failed to get container status \"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab\": rpc error: code = NotFound desc = could not find container \"eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab\": container with ID starting with eb0d7fd44cf2cccffc7c7460cc6e18eb2b77707b489b08bf8e7f0de0603337ab not found: ID does not exist" Jul 15 08:30:19.752622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.752546 2581 scope.go:117] "RemoveContainer" containerID="8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e" Jul 15 08:30:19.752860 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:30:19.752840 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e\": container with ID starting with 8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e not found: ID does not exist" containerID="8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e" Jul 15 08:30:19.752923 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.752874 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e"} err="failed to get container status \"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e\": rpc error: code = NotFound desc = could not find container \"8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e\": container with ID starting with 8cd6d1091d1f5585cdc76e11ac05067b0b88bc2ed3e41e1d63381bf3226a406e not found: ID does not exist" Jul 15 08:30:19.754726 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.754705 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:30:19.758480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:19.758457 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4dbdead5-openshift-default-74ffd54f97-7dz5w"] Jul 15 08:30:21.351496 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:21.351464 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" path="/var/lib/kubelet/pods/daf0ecfd-6678-484d-9bcb-a549baab0dc2/volumes" Jul 15 08:30:25.768824 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.768790 2581 generic.go:356] "Generic (PLEG): container finished" podID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerID="1fb5444d149bf884f322dfec32c33bfd89e267570c6fa8cc6b83f60c0e519ce3" exitCode=0 Jul 15 08:30:25.769204 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.768863 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerDied","Data":"1fb5444d149bf884f322dfec32c33bfd89e267570c6fa8cc6b83f60c0e519ce3"} Jul 15 08:30:25.769411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.769393 2581 scope.go:117] "RemoveContainer" containerID="1fb5444d149bf884f322dfec32c33bfd89e267570c6fa8cc6b83f60c0e519ce3" Jul 15 08:30:25.770461 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.770427 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:25.770575 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.770515 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:25.770575 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:25.770563 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:30:26.780285 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:26.780238 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerStarted","Data":"bb6ed26a975e1bcbfeb68a71f9d8597a523afc203616dc33be06fc1732894b6d"} Jul 15 08:30:27.450115 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:27.450084 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:27.451152 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:27.451124 2581 patch_prober.go:28] interesting pod/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 15 08:30:27.451280 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:27.451180 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 15 08:30:28.449867 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:28.449836 2581 patch_prober.go:28] interesting pod/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 15 08:30:28.450222 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:28.449884 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 15 08:30:29.450306 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:29.450230 2581 patch_prober.go:28] interesting pod/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 15 08:30:29.450306 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:29.450292 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 15 08:30:30.465117 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:30.465086 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:30.465595 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:30.465375 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:30.466122 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:30.466101 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:30:37.189234 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:37.189205 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:30:37.194751 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:30:37.194734 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:31:45.016456 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:45.016421 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:31:45.017060 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:45.016674 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" containerID="cri-o://bb6ed26a975e1bcbfeb68a71f9d8597a523afc203616dc33be06fc1732894b6d" gracePeriod=30 Jul 15 08:31:45.467550 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:45.467513 2581 patch_prober.go:28] interesting pod/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:31:45.467716 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:45.467588 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:31:50.190218 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.190191 2581 generic.go:356] "Generic (PLEG): container finished" podID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerID="bb6ed26a975e1bcbfeb68a71f9d8597a523afc203616dc33be06fc1732894b6d" exitCode=0 Jul 15 08:31:50.190653 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.190261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerDied","Data":"bb6ed26a975e1bcbfeb68a71f9d8597a523afc203616dc33be06fc1732894b6d"} Jul 15 08:31:50.190653 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.190323 2581 scope.go:117] "RemoveContainer" containerID="1fb5444d149bf884f322dfec32c33bfd89e267570c6fa8cc6b83f60c0e519ce3" Jul 15 08:31:50.275302 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.275281 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:31:50.413159 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413130 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413159 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413211 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413284 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413308 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413379 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413410 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7lg8\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413436 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413466 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs\") pod \"f7bff8ed-fa5c-44ca-9386-4a037149016b\" (UID: \"f7bff8ed-fa5c-44ca-9386-4a037149016b\") " Jul 15 08:31:50.413607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413507 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:31:50.413823 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413777 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.413823 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413799 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:31:50.413924 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413827 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:31:50.414012 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.413983 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:31:50.414238 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.414216 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:31:50.415471 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.415442 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:31:50.415580 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.415535 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:31:50.415882 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.415855 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:31:50.415952 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.415873 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8" (OuterVolumeSpecName: "kube-api-access-g7lg8") pod "f7bff8ed-fa5c-44ca-9386-4a037149016b" (UID: "f7bff8ed-fa5c-44ca-9386-4a037149016b"). InnerVolumeSpecName "kube-api-access-g7lg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:31:50.515010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.514977 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515010 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7lg8\" (UniqueName: \"kubernetes.io/projected/f7bff8ed-fa5c-44ca-9386-4a037149016b-kube-api-access-g7lg8\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515025 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7bff8ed-fa5c-44ca-9386-4a037149016b-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515038 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515050 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515061 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515072 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:50.515255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:50.515099 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7bff8ed-fa5c-44ca-9386-4a037149016b-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:31:51.196710 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.196677 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" event={"ID":"f7bff8ed-fa5c-44ca-9386-4a037149016b","Type":"ContainerDied","Data":"47a58c39d9e25e9ac9517c20c211826411996b401f0442343ee99966fa25553f"} Jul 15 08:31:51.196710 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.196706 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp" Jul 15 08:31:51.197207 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.196737 2581 scope.go:117] "RemoveContainer" containerID="bb6ed26a975e1bcbfeb68a71f9d8597a523afc203616dc33be06fc1732894b6d" Jul 15 08:31:51.227476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.227447 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:31:51.236984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.236961 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-93b7ed10-openshift-default-7474d76f8c-6qcvp"] Jul 15 08:31:51.357751 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:31:51.357707 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" path="/var/lib/kubelet/pods/f7bff8ed-fa5c-44ca-9386-4a037149016b/volumes" Jul 15 08:33:26.498915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.498884 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499230 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499240 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499250 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499256 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499264 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499269 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499328 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:33:26.499441 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.499338 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:33:26.502288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.502265 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.506899 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.506874 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-13831f8c-openshift-default-dockercfg-4mkdf\"" Jul 15 08:33:26.525341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.525316 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:33:26.612631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612648 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612673 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612826 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tkt\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.612943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.613040 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.612969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714273 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714325 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714481 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26tkt\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714512 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714808 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714758 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714969 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714832 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.714969 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.714876 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.715075 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.715006 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.715167 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.715136 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.716863 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.716844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.717063 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.717042 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.724568 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.724525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.724704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.724593 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tkt\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt\") pod \"e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.811967 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.811891 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:26.961217 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.961183 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:33:26.964187 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:33:26.964157 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2e808e_fff0_452b_a276_569e5ccf953d.slice/crio-230bba9b31fcd92a4aa1a3fed705b00dfcba27366861cea22c899f1bae0ba9ae WatchSource:0}: Error finding container 230bba9b31fcd92a4aa1a3fed705b00dfcba27366861cea22c899f1bae0ba9ae: Status 404 returned error can't find the container with id 230bba9b31fcd92a4aa1a3fed705b00dfcba27366861cea22c899f1bae0ba9ae Jul 15 08:33:26.966570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.966535 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:26.966683 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.966613 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:26.966683 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:26.966667 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:27.661719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:27.661685 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerStarted","Data":"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921"} Jul 15 08:33:27.662111 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:27.661740 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerStarted","Data":"230bba9b31fcd92a4aa1a3fed705b00dfcba27366861cea22c899f1bae0ba9ae"} Jul 15 08:33:27.698526 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:27.698478 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podStartSLOduration=1.698461847 podStartE2EDuration="1.698461847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:33:27.698253856 +0000 UTC m=+1971.089658478" watchObservedRunningTime="2026-07-15 08:33:27.698461847 +0000 UTC m=+1971.089866470" Jul 15 08:33:27.812617 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:27.812581 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:28.817525 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:28.817493 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:29.673140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:29.673056 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:29.673829 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:29.673808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:42.736278 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.736245 2581 generic.go:356] "Generic (PLEG): container finished" podID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerID="f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921" exitCode=0 Jul 15 08:33:42.736701 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.736326 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerDied","Data":"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921"} Jul 15 08:33:42.736752 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.736712 2581 scope.go:117] "RemoveContainer" containerID="f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921" Jul 15 08:33:42.737671 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.737644 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:42.737745 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.737728 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:42.737826 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:42.737805 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:33:43.744025 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:43.743989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerStarted","Data":"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7"} Jul 15 08:33:43.812326 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:43.812290 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:43.813247 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:43.813221 2581 patch_prober.go:28] interesting pod/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 08:33:43.813304 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:43.813268 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 08:33:44.673375 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:44.673314 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:44.812595 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:44.812560 2581 patch_prober.go:28] interesting pod/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 08:33:44.812965 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:44.812616 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 08:33:45.812433 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:45.812401 2581 patch_prober.go:28] interesting pod/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 08:33:45.812601 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:45.812448 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 08:33:46.812592 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:46.812557 2581 patch_prober.go:28] interesting pod/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 15 08:33:46.813049 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:46.812617 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 15 08:33:47.815845 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:47.815808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:33:47.816773 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:33:47.816753 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:34:08.456015 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:08.455980 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:34:08.456710 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:08.456261 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" containerID="cri-o://c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7" gracePeriod=30 Jul 15 08:34:13.698908 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.698885 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:34:13.852292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852265 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852303 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852319 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852347 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852440 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852476 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852539 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852499 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26tkt\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852550 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852573 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo\") pod \"ae2e808e-fff0-452b-a276-569e5ccf953d\" (UID: \"ae2e808e-fff0-452b-a276-569e5ccf953d\") " Jul 15 08:34:13.852855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852810 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:34:13.852855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852806 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:34:13.853063 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.852907 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:34:13.853119 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.853069 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.853119 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.853089 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae2e808e-fff0-452b-a276-569e5ccf953d-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.853119 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.853104 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.853254 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.853161 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:34:13.853254 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.853199 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:34:13.854604 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.854576 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:34:13.854742 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.854720 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:34:13.855003 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.854970 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:34:13.855059 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.855013 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt" (OuterVolumeSpecName: "kube-api-access-26tkt") pod "ae2e808e-fff0-452b-a276-569e5ccf953d" (UID: "ae2e808e-fff0-452b-a276-569e5ccf953d"). InnerVolumeSpecName "kube-api-access-26tkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:34:13.895711 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.895684 2581 generic.go:356] "Generic (PLEG): container finished" podID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerID="c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7" exitCode=0 Jul 15 08:34:13.895820 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.895725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerDied","Data":"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7"} Jul 15 08:34:13.895820 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.895731 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" Jul 15 08:34:13.895820 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.895788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7" event={"ID":"ae2e808e-fff0-452b-a276-569e5ccf953d","Type":"ContainerDied","Data":"230bba9b31fcd92a4aa1a3fed705b00dfcba27366861cea22c899f1bae0ba9ae"} Jul 15 08:34:13.895820 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.895810 2581 scope.go:117] "RemoveContainer" containerID="c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7" Jul 15 08:34:13.904525 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.904511 2581 scope.go:117] "RemoveContainer" containerID="f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921" Jul 15 08:34:13.911918 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.911900 2581 scope.go:117] "RemoveContainer" containerID="c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7" Jul 15 08:34:13.912155 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:34:13.912137 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7\": container with ID starting with c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7 not found: ID does not exist" containerID="c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7" Jul 15 08:34:13.912203 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.912162 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7"} err="failed to get container status \"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7\": rpc error: code = NotFound desc = could not find container \"c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7\": container with ID starting with c94605e1a6202ace197edb141c4b2633a049ff7247a9b24a1ee7ad48e1721bd7 not found: ID does not exist" Jul 15 08:34:13.912203 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.912192 2581 scope.go:117] "RemoveContainer" containerID="f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921" Jul 15 08:34:13.912439 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:34:13.912421 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921\": container with ID starting with f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921 not found: ID does not exist" containerID="f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921" Jul 15 08:34:13.912499 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.912447 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921"} err="failed to get container status \"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921\": rpc error: code = NotFound desc = could not find container \"f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921\": container with ID starting with f12cf8c57fe0b27bb275084527d821968944fe890d0665d2e66c298edac60921 not found: ID does not exist" Jul 15 08:34:13.937705 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.937663 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:34:13.939789 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.939770 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-13831f8c-openshift-default-5fd7b875f9-9wpj7"] Jul 15 08:34:13.953622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953608 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.953688 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953623 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.953688 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953632 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26tkt\" (UniqueName: \"kubernetes.io/projected/ae2e808e-fff0-452b-a276-569e5ccf953d-kube-api-access-26tkt\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.953688 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953641 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.953688 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953650 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:13.953688 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:13.953658 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae2e808e-fff0-452b-a276-569e5ccf953d-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:34:15.352257 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:15.352223 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" path="/var/lib/kubelet/pods/ae2e808e-fff0-452b-a276-569e5ccf953d/volumes" Jul 15 08:34:31.274432 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.274391 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:34:31.274995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.274975 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:34:31.275076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.274999 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:34:31.275076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275018 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.275076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275028 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.275076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275040 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.275076 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275045 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.275292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275129 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7bff8ed-fa5c-44ca-9386-4a037149016b" containerName="istio-proxy" Jul 15 08:34:31.275292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275141 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="daf0ecfd-6678-484d-9bcb-a549baab0dc2" containerName="istio-proxy" Jul 15 08:34:31.275292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275149 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.275292 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.275157 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae2e808e-fff0-452b-a276-569e5ccf953d" containerName="istio-proxy" Jul 15 08:34:31.278723 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.278705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.282798 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.282780 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2082f65c-openshift-default-dockercfg-8wtbg\"" Jul 15 08:34:31.293652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.293625 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:34:31.298169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298200 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46kvw\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298436 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298564 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298564 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298564 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.298564 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.298535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.399870 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399833 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399878 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399905 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399938 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.399975 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46kvw\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400305 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400321 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.400692 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.401009 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.400986 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.402413 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.402396 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.402879 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.402858 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.414056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.414031 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.415158 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.415138 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46kvw\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw\") pod \"e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.589107 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.589080 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:31.735274 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.735250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:34:31.737474 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:34:31.737448 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7158c3_18fa_4ceb_8724_c6fd5945caca.slice/crio-34af1dd6cc744d7ae0858544b0f37522ba25d3a6bfaa3483b71e038ca21652ec WatchSource:0}: Error finding container 34af1dd6cc744d7ae0858544b0f37522ba25d3a6bfaa3483b71e038ca21652ec: Status 404 returned error can't find the container with id 34af1dd6cc744d7ae0858544b0f37522ba25d3a6bfaa3483b71e038ca21652ec Jul 15 08:34:31.739543 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.739513 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:31.739631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.739581 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:31.739631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.739611 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:31.988167 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.988081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerStarted","Data":"1d60264c8d7124499c29b19760a0b916c82196e97c111459e363ff88a75ed9a9"} Jul 15 08:34:31.988167 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:31.988142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerStarted","Data":"34af1dd6cc744d7ae0858544b0f37522ba25d3a6bfaa3483b71e038ca21652ec"} Jul 15 08:34:32.012499 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:32.012445 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podStartSLOduration=1.012424773 podStartE2EDuration="1.012424773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:34:32.009435265 +0000 UTC m=+2035.400839909" watchObservedRunningTime="2026-07-15 08:34:32.012424773 +0000 UTC m=+2035.403829400" Jul 15 08:34:32.589376 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:32.589324 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:32.595521 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:32.595496 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:32.994228 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:32.994136 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:32.995189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:32.995169 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:47.061073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.061039 2581 generic.go:356] "Generic (PLEG): container finished" podID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerID="1d60264c8d7124499c29b19760a0b916c82196e97c111459e363ff88a75ed9a9" exitCode=0 Jul 15 08:34:47.061634 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.061113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerDied","Data":"1d60264c8d7124499c29b19760a0b916c82196e97c111459e363ff88a75ed9a9"} Jul 15 08:34:47.061703 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.061644 2581 scope.go:117] "RemoveContainer" containerID="1d60264c8d7124499c29b19760a0b916c82196e97c111459e363ff88a75ed9a9" Jul 15 08:34:47.062655 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.062628 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:47.062753 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.062693 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:47.062753 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.062724 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:34:47.994926 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:47.994883 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:48.068220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:48.068179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerStarted","Data":"5b21311a32d7b4661e916db0611ec88594dc9b30b7454cf206b37ac03a59a305"} Jul 15 08:34:48.591382 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:48.591325 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:48.591574 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:48.591480 2581 patch_prober.go:28] interesting pod/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 08:34:48.591574 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:48.591532 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 08:34:49.590518 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:49.590474 2581 patch_prober.go:28] interesting pod/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 08:34:49.591041 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:49.590564 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 08:34:50.589870 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:50.589819 2581 patch_prober.go:28] interesting pod/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 08:34:50.590088 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:50.589884 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 08:34:51.590423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:51.590386 2581 patch_prober.go:28] interesting pod/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 15 08:34:51.591015 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:51.590981 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 15 08:34:52.593681 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:52.593655 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:52.594074 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:52.593902 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:34:52.594641 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:34:52.594620 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:35:37.218862 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:35:37.218832 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:35:37.223943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:35:37.223926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:36:37.435763 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:37.435730 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:36:37.436309 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:37.435990 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" containerID="cri-o://5b21311a32d7b4661e916db0611ec88594dc9b30b7454cf206b37ac03a59a305" gracePeriod=30 Jul 15 08:36:37.596326 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:37.596286 2581 patch_prober.go:28] interesting pod/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:36:37.596507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:37.596344 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:36:42.630072 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.630038 2581 generic.go:356] "Generic (PLEG): container finished" podID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerID="5b21311a32d7b4661e916db0611ec88594dc9b30b7454cf206b37ac03a59a305" exitCode=0 Jul 15 08:36:42.630553 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.630176 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerDied","Data":"5b21311a32d7b4661e916db0611ec88594dc9b30b7454cf206b37ac03a59a305"} Jul 15 08:36:42.630553 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.630236 2581 scope.go:117] "RemoveContainer" containerID="1d60264c8d7124499c29b19760a0b916c82196e97c111459e363ff88a75ed9a9" Jul 15 08:36:42.706906 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.706840 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:36:42.868413 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868381 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868413 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868417 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868471 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868510 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868643 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868542 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868652 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:36:42.868858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868662 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46kvw\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868720 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868762 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy\") pod \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\" (UID: \"0e7158c3-18fa-4ceb-8724-c6fd5945caca\") " Jul 15 08:36:42.868858 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868823 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:36:42.869126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868913 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:36:42.869126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.868978 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:36:42.869126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.869092 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.869126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.869111 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.869126 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.869126 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.869323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.869139 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.869323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.869240 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:36:42.871063 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.871036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:36:42.871189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.871066 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:36:42.871189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.871144 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:36:42.871300 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.871278 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw" (OuterVolumeSpecName: "kube-api-access-46kvw") pod "0e7158c3-18fa-4ceb-8724-c6fd5945caca" (UID: "0e7158c3-18fa-4ceb-8724-c6fd5945caca"). InnerVolumeSpecName "kube-api-access-46kvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:36:42.970185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.970113 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46kvw\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-kube-api-access-46kvw\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.970185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.970138 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.970185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.970148 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.970185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.970156 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:42.970185 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:42.970164 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e7158c3-18fa-4ceb-8724-c6fd5945caca-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:36:43.636226 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:43.636136 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" event={"ID":"0e7158c3-18fa-4ceb-8724-c6fd5945caca","Type":"ContainerDied","Data":"34af1dd6cc744d7ae0858544b0f37522ba25d3a6bfaa3483b71e038ca21652ec"} Jul 15 08:36:43.636226 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:43.636164 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr" Jul 15 08:36:43.636226 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:43.636192 2581 scope.go:117] "RemoveContainer" containerID="5b21311a32d7b4661e916db0611ec88594dc9b30b7454cf206b37ac03a59a305" Jul 15 08:36:43.654632 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:43.654610 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:36:43.660741 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:43.660719 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2082f65c-openshift-default-868c7b4fc9-kxqmr"] Jul 15 08:36:45.351482 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:36:45.351442 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" path="/var/lib/kubelet/pods/0e7158c3-18fa-4ceb-8724-c6fd5945caca/volumes" Jul 15 08:37:32.841884 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.841851 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842188 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842198 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842212 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842217 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842284 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.842396 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.842291 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7158c3-18fa-4ceb-8724-c6fd5945caca" containerName="istio-proxy" Jul 15 08:37:32.845557 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.845540 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.853876 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.853853 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f6b673-openshift-default-dockercfg-vs9cp\"" Jul 15 08:37:32.857869 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.857821 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:37:32.989400 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989344 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989400 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989618 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989618 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989461 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vtdf\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989618 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989618 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989638 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:32.989791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:32.989700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090631 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.090761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.090732 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091084 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091054 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091084 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091068 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091141 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vtdf\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091148 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091344 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091214 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091545 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091519 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.091949 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.091902 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.093010 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.092988 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.093243 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.093224 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.101189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.101159 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.101637 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.101617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vtdf\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf\") pod \"e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.162959 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.162933 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:33.313578 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.313555 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:37:33.315768 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:37:33.315738 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267f97b0_5fba_413b_a934_123c16865976.slice/crio-baca1d669d97e95a53b131bb50b76862e3862c46ed9999c8eb68eef372751395 WatchSource:0}: Error finding container baca1d669d97e95a53b131bb50b76862e3862c46ed9999c8eb68eef372751395: Status 404 returned error can't find the container with id baca1d669d97e95a53b131bb50b76862e3862c46ed9999c8eb68eef372751395 Jul 15 08:37:33.317831 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.317791 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:33.317922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.317874 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:33.317922 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.317910 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:33.875599 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.875561 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerStarted","Data":"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e"} Jul 15 08:37:33.875971 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.875623 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerStarted","Data":"baca1d669d97e95a53b131bb50b76862e3862c46ed9999c8eb68eef372751395"} Jul 15 08:37:33.900186 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:33.900144 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podStartSLOduration=1.900130257 podStartE2EDuration="1.900130257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:37:33.898785258 +0000 UTC m=+2217.290189902" watchObservedRunningTime="2026-07-15 08:37:33.900130257 +0000 UTC m=+2217.291534880" Jul 15 08:37:34.163714 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:34.163642 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:34.168865 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:34.168841 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:34.880426 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:34.880396 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:34.881207 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:34.881189 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:41.071672 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.070775 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:37:41.076537 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.076512 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.080572 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.080556 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4c59e6-openshift-default-dockercfg-2pqvv\"" Jul 15 08:37:41.089398 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.089344 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:37:41.161640 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.161608 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.161905 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.161877 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.161929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.161957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.161981 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.162042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.162072 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.162111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.162266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.162153 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qnt5\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263761 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.263797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qnt5\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.263860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.264001 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264181 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.264040 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264328 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.264187 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264328 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.264265 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.264632 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.264606 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.266381 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.266333 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.266504 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.266482 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.272784 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.272754 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.273037 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.273014 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qnt5\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5\") pod \"e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.386843 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.386813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:41.528608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.528584 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:37:41.530371 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:37:41.530331 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92be3d16_0ecc_4b97_8924_9905a14cc37f.slice/crio-7065a8cb98feb6c747bc648e07a50f3f04098a8e5bd45ed7823a89a42ab43a91 WatchSource:0}: Error finding container 7065a8cb98feb6c747bc648e07a50f3f04098a8e5bd45ed7823a89a42ab43a91: Status 404 returned error can't find the container with id 7065a8cb98feb6c747bc648e07a50f3f04098a8e5bd45ed7823a89a42ab43a91 Jul 15 08:37:41.532622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.532584 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:41.532719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.532651 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:41.532719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.532682 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:41.915168 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.915132 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerStarted","Data":"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01"} Jul 15 08:37:41.915341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.915184 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerStarted","Data":"7065a8cb98feb6c747bc648e07a50f3f04098a8e5bd45ed7823a89a42ab43a91"} Jul 15 08:37:41.935701 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:41.935650 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podStartSLOduration=0.93563339 podStartE2EDuration="935.63339ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:37:41.934307817 +0000 UTC m=+2225.325712441" watchObservedRunningTime="2026-07-15 08:37:41.93563339 +0000 UTC m=+2225.327038017" Jul 15 08:37:42.387321 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:42.387289 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:42.391914 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:42.391890 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:42.920659 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:42.920632 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:42.921467 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:42.921444 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:37:50.922547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:50.922500 2581 patch_prober.go:28] interesting pod/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 08:37:50.923057 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:50.922588 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 08:37:51.967661 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.967566 2581 generic.go:356] "Generic (PLEG): container finished" podID="267f97b0-5fba-413b-a934-123c16865976" containerID="f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e" exitCode=0 Jul 15 08:37:51.967661 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.967641 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerDied","Data":"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e"} Jul 15 08:37:51.968507 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.968115 2581 scope.go:117] "RemoveContainer" containerID="f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e" Jul 15 08:37:51.969393 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.969336 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:51.969482 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.969444 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:51.969531 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:51.969489 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:37:52.996412 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:52.994613 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerStarted","Data":"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d"} Jul 15 08:37:53.164165 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:53.164122 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:53.164437 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:53.164339 2581 patch_prober.go:28] interesting pod/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 08:37:53.164675 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:53.164648 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 08:37:54.163348 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:54.163312 2581 patch_prober.go:28] interesting pod/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 08:37:54.163854 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:54.163399 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 08:37:55.163765 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:55.163730 2581 patch_prober.go:28] interesting pod/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 08:37:55.164149 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:55.163781 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 08:37:56.164288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:56.164251 2581 patch_prober.go:28] interesting pod/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 15 08:37:56.164857 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:56.164306 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 15 08:37:57.172599 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:57.172562 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:57.173907 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:57.173876 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:57.180410 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:57.180384 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:37:58.962537 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:58.962498 2581 patch_prober.go:28] interesting pod/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 08:37:58.962940 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:37:58.962561 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 08:38:01.048924 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.048891 2581 generic.go:356] "Generic (PLEG): container finished" podID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerID="7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01" exitCode=0 Jul 15 08:38:01.049341 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.048964 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerDied","Data":"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01"} Jul 15 08:38:01.049410 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.049389 2581 scope.go:117] "RemoveContainer" containerID="7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01" Jul 15 08:38:01.050323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.050298 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:38:01.050403 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.050387 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:38:01.050448 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:01.050424 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:38:02.057278 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:02.057231 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerStarted","Data":"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7"} Jul 15 08:38:02.388006 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:02.387959 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:38:02.388323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:02.388087 2581 patch_prober.go:28] interesting pod/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 15 08:38:02.388522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:02.388487 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 15 08:38:03.387741 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:03.387702 2581 patch_prober.go:28] interesting pod/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 15 08:38:03.388252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:03.387769 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 15 08:38:04.387784 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:04.387744 2581 patch_prober.go:28] interesting pod/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 15 08:38:04.388399 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:04.388343 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 15 08:38:05.388229 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:05.388189 2581 patch_prober.go:28] interesting pod/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 15 08:38:05.388835 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:05.388802 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 15 08:38:06.390736 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:06.390707 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:38:06.391121 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:06.391055 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:38:06.391776 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:38:06.391758 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:39:19.514009 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:19.513973 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:39:19.514668 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:19.514171 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" containerID="cri-o://7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d" gracePeriod=30 Jul 15 08:39:24.759551 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.759524 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:39:24.889585 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889552 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889585 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889590 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889620 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889646 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889672 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889706 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889740 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.889804 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.890122 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889807 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vtdf\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf\") pod \"267f97b0-5fba-413b-a934-123c16865976\" (UID: \"267f97b0-5fba-413b-a934-123c16865976\") " Jul 15 08:39:24.890122 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.889837 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:24.890235 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.890188 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:24.890235 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.890207 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.890455 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.890414 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:24.890455 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.890426 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data" (OuterVolumeSpecName: "istio-data") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:24.890617 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.890593 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:39:24.892009 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.891981 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:39:24.892116 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.892042 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token" (OuterVolumeSpecName: "istio-token") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:39:24.892336 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.892320 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:24.892492 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.892339 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf" (OuterVolumeSpecName: "kube-api-access-5vtdf") pod "267f97b0-5fba-413b-a934-123c16865976" (UID: "267f97b0-5fba-413b-a934-123c16865976"). InnerVolumeSpecName "kube-api-access-5vtdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:39:24.991112 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991091 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991112 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991112 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991120 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991128 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f97b0-5fba-413b-a934-123c16865976-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991136 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991143 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f97b0-5fba-413b-a934-123c16865976-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991152 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f97b0-5fba-413b-a934-123c16865976-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:24.991263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:24.991160 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vtdf\" (UniqueName: \"kubernetes.io/projected/267f97b0-5fba-413b-a934-123c16865976-kube-api-access-5vtdf\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:25.489404 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.489371 2581 generic.go:356] "Generic (PLEG): container finished" podID="267f97b0-5fba-413b-a934-123c16865976" containerID="7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d" exitCode=0 Jul 15 08:39:25.489576 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.489418 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" Jul 15 08:39:25.489576 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.489445 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerDied","Data":"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d"} Jul 15 08:39:25.489576 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.489498 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p" event={"ID":"267f97b0-5fba-413b-a934-123c16865976","Type":"ContainerDied","Data":"baca1d669d97e95a53b131bb50b76862e3862c46ed9999c8eb68eef372751395"} Jul 15 08:39:25.489576 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.489521 2581 scope.go:117] "RemoveContainer" containerID="7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d" Jul 15 08:39:25.497944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.497930 2581 scope.go:117] "RemoveContainer" containerID="f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e" Jul 15 08:39:25.505329 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.505305 2581 scope.go:117] "RemoveContainer" containerID="7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d" Jul 15 08:39:25.505606 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:39:25.505588 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d\": container with ID starting with 7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d not found: ID does not exist" containerID="7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d" Jul 15 08:39:25.505659 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.505613 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d"} err="failed to get container status \"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d\": rpc error: code = NotFound desc = could not find container \"7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d\": container with ID starting with 7d6802b2ac3cf0611e37cd01d904ac32703e642a436df46585c3899ba62df78d not found: ID does not exist" Jul 15 08:39:25.505659 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.505628 2581 scope.go:117] "RemoveContainer" containerID="f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e" Jul 15 08:39:25.505856 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:39:25.505827 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e\": container with ID starting with f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e not found: ID does not exist" containerID="f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e" Jul 15 08:39:25.505936 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.505860 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e"} err="failed to get container status \"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e\": rpc error: code = NotFound desc = could not find container \"f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e\": container with ID starting with f1e3149df75d2a026b3176227feaf68392d5f5a077a3e6f800800b763c54578e not found: ID does not exist" Jul 15 08:39:25.518746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.518725 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:39:25.526073 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:25.526052 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f6b673-openshift-default-df5b67c6f-7pk2p"] Jul 15 08:39:27.352150 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:27.352111 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="267f97b0-5fba-413b-a934-123c16865976" path="/var/lib/kubelet/pods/267f97b0-5fba-413b-a934-123c16865976/volumes" Jul 15 08:39:45.748170 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:45.748138 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:39:45.748657 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:45.748321 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" containerID="cri-o://eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7" gracePeriod=30 Jul 15 08:39:46.694573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.694538 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:39:46.695059 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.695044 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:46.695106 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.695063 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:46.695188 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.695177 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:46.695227 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.695190 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:46.698528 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.698509 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.701429 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.701411 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ec03f1-openshift-default-dockercfg-gqmnj\"" Jul 15 08:39:46.713574 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.713550 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:39:46.774605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.774568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775101 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzd5m\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775293 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775514 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775665 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775650 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775791 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.775934 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.775920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.776056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.776043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.776169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.776156 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzd5m\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877432 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877491 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877753 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877753 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877753 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877639 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.877912 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.877786 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.878140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.878098 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.878140 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.878118 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.878340 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.878225 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.878340 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.878252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.879855 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.879838 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.880090 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.880069 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.895464 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.895445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzd5m\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:46.898802 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:46.898777 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token\") pod \"e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:47.009339 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.009256 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:47.197041 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.197018 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:39:47.199032 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:39:47.199008 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbc9730_5907_478b_b2fc_5085a783ffaa.slice/crio-896bb9e5f8a0ba6731b45ababeeb77b366024496c5805b5c73cc00def0c3d824 WatchSource:0}: Error finding container 896bb9e5f8a0ba6731b45ababeeb77b366024496c5805b5c73cc00def0c3d824: Status 404 returned error can't find the container with id 896bb9e5f8a0ba6731b45ababeeb77b366024496c5805b5c73cc00def0c3d824 Jul 15 08:39:47.201112 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.201076 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:47.201213 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.201155 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:47.201213 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.201204 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:47.596437 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.596390 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerStarted","Data":"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235"} Jul 15 08:39:47.596653 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.596473 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerStarted","Data":"896bb9e5f8a0ba6731b45ababeeb77b366024496c5805b5c73cc00def0c3d824"} Jul 15 08:39:47.628962 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:47.628915 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podStartSLOduration=1.628899808 podStartE2EDuration="1.628899808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:39:47.627822 +0000 UTC m=+2351.019226623" watchObservedRunningTime="2026-07-15 08:39:47.628899808 +0000 UTC m=+2351.020304477" Jul 15 08:39:48.009983 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:48.009906 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:48.015121 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:48.015095 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:48.601645 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:48.601618 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:48.602586 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:48.602569 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:39:51.008755 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.008701 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:39:51.112907 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.112878 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.112921 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.112942 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.112968 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113100 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qnt5\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113160 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113203 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113253 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113287 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket\") pod \"92be3d16-0ecc-4b97-8924-9905a14cc37f\" (UID: \"92be3d16-0ecc-4b97-8924-9905a14cc37f\") " Jul 15 08:39:51.113419 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113300 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:39:51.113677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113487 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:51.113677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113605 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:51.113677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113623 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92be3d16-0ecc-4b97-8924-9905a14cc37f-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.113677 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113642 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.113897 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.113740 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:51.114233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.114203 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data" (OuterVolumeSpecName: "istio-data") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:51.115272 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.115241 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5" (OuterVolumeSpecName: "kube-api-access-2qnt5") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "kube-api-access-2qnt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:39:51.115386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.115253 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:39:51.115386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.115276 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:39:51.115386 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.115348 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token" (OuterVolumeSpecName: "istio-token") pod "92be3d16-0ecc-4b97-8924-9905a14cc37f" (UID: "92be3d16-0ecc-4b97-8924-9905a14cc37f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:39:51.214233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214199 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214228 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214233 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214238 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qnt5\" (UniqueName: \"kubernetes.io/projected/92be3d16-0ecc-4b97-8924-9905a14cc37f-kube-api-access-2qnt5\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214477 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214248 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214477 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214256 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214477 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214268 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.214477 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.214280 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92be3d16-0ecc-4b97-8924-9905a14cc37f-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:39:51.616882 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.616799 2581 generic.go:356] "Generic (PLEG): container finished" podID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerID="eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7" exitCode=0 Jul 15 08:39:51.616882 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.616840 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" Jul 15 08:39:51.617110 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.616881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerDied","Data":"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7"} Jul 15 08:39:51.617110 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.616941 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp" event={"ID":"92be3d16-0ecc-4b97-8924-9905a14cc37f","Type":"ContainerDied","Data":"7065a8cb98feb6c747bc648e07a50f3f04098a8e5bd45ed7823a89a42ab43a91"} Jul 15 08:39:51.617110 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.616969 2581 scope.go:117] "RemoveContainer" containerID="eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7" Jul 15 08:39:51.632675 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.632590 2581 scope.go:117] "RemoveContainer" containerID="7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01" Jul 15 08:39:51.639971 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.639946 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:39:51.640547 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.640533 2581 scope.go:117] "RemoveContainer" containerID="eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7" Jul 15 08:39:51.640798 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:39:51.640780 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7\": container with ID starting with eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7 not found: ID does not exist" containerID="eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7" Jul 15 08:39:51.640846 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.640806 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7"} err="failed to get container status \"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7\": rpc error: code = NotFound desc = could not find container \"eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7\": container with ID starting with eecf28d5e38fa392d3a3124f7a2429acccaa5a0270e0995c9a33ee4dae5ba5e7 not found: ID does not exist" Jul 15 08:39:51.640846 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.640823 2581 scope.go:117] "RemoveContainer" containerID="7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01" Jul 15 08:39:51.641034 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:39:51.641021 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01\": container with ID starting with 7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01 not found: ID does not exist" containerID="7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01" Jul 15 08:39:51.641080 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.641038 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01"} err="failed to get container status \"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01\": rpc error: code = NotFound desc = could not find container \"7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01\": container with ID starting with 7a963e4f883f9cc360377c76af8ebe8d4a50431d94bfdc31cb67e995b9544e01 not found: ID does not exist" Jul 15 08:39:51.644746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:51.644725 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4c59e6-openshift-default-7df6f46f95-llrfp"] Jul 15 08:39:53.351847 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:53.351815 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" path="/var/lib/kubelet/pods/92be3d16-0ecc-4b97-8924-9905a14cc37f/volumes" Jul 15 08:39:55.021834 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.021796 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022152 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022163 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022171 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022178 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022202 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022208 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="267f97b0-5fba-413b-a934-123c16865976" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022262 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.022286 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.022271 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="92be3d16-0ecc-4b97-8924-9905a14cc37f" containerName="istio-proxy" Jul 15 08:39:55.026632 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.026613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.029508 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.029484 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7a0e41-openshift-default-dockercfg-xgkd9\"" Jul 15 08:39:55.037230 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.037208 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:39:55.147754 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.147725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.147943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.147780 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.147943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.147851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.147943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.147894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.148090 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.147970 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.148090 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.148016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.148090 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.148039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.148232 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.148105 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkfz4\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.148232 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.148140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.248923 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.248889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249091 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249169 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249152 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249306 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249393 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249311 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249533 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkfz4\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249631 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249694 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249622 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249755 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249734 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.249857 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.249842 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.251476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.251450 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.252047 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.252030 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.257032 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.257005 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.257195 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.257173 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkfz4\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4\") pod \"e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.337604 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.337577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:55.497956 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.497924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:39:55.502074 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:39:55.502033 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5938f3_8831_43de_ba53_d8ebc1e7cbc4.slice/crio-51c5181e0592d16a123ed45a2433a24308895d534125862e96a502aed34447ac WatchSource:0}: Error finding container 51c5181e0592d16a123ed45a2433a24308895d534125862e96a502aed34447ac: Status 404 returned error can't find the container with id 51c5181e0592d16a123ed45a2433a24308895d534125862e96a502aed34447ac Jul 15 08:39:55.504621 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.504581 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:55.504734 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.504663 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:55.504734 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.504706 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:39:55.640812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.640586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerStarted","Data":"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b"} Jul 15 08:39:55.640812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:55.640650 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerStarted","Data":"51c5181e0592d16a123ed45a2433a24308895d534125862e96a502aed34447ac"} Jul 15 08:39:56.338462 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:56.338432 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:56.343423 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:56.343397 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:56.377657 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:56.377593 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podStartSLOduration=2.377555181 podStartE2EDuration="2.377555181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:39:55.67384433 +0000 UTC m=+2359.065248948" watchObservedRunningTime="2026-07-15 08:39:56.377555181 +0000 UTC m=+2359.768959805" Jul 15 08:39:56.645628 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:56.645549 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:39:56.646574 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:39:56.646558 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:02.680266 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.680229 2581 generic.go:356] "Generic (PLEG): container finished" podID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerID="2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235" exitCode=0 Jul 15 08:40:02.680817 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.680312 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerDied","Data":"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235"} Jul 15 08:40:02.680891 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.680819 2581 scope.go:117] "RemoveContainer" containerID="2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235" Jul 15 08:40:02.681891 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.681861 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:02.681988 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.681925 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:02.681988 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:02.681963 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:03.602465 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:03.602431 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:03.691930 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:03.691886 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerStarted","Data":"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86"} Jul 15 08:40:04.010939 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:04.010836 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:04.011123 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:04.011002 2581 patch_prober.go:28] interesting pod/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 08:40:04.011123 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:04.011047 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 08:40:05.010788 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:05.010740 2581 patch_prober.go:28] interesting pod/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 08:40:05.011311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:05.010814 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 08:40:06.010699 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:06.010653 2581 patch_prober.go:28] interesting pod/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 08:40:06.010920 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:06.010740 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 08:40:07.009892 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:07.009844 2581 patch_prober.go:28] interesting pod/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 15 08:40:07.010117 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:07.009916 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 15 08:40:08.013925 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:08.013890 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:08.014415 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:08.014255 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:08.015265 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:08.015244 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:10.735353 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.735321 2581 generic.go:356] "Generic (PLEG): container finished" podID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerID="a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b" exitCode=0 Jul 15 08:40:10.735895 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.735398 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerDied","Data":"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b"} Jul 15 08:40:10.735978 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.735904 2581 scope.go:117] "RemoveContainer" containerID="a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b" Jul 15 08:40:10.736886 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.736854 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:10.736943 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.736923 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:10.736983 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:10.736956 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:40:11.646129 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:11.646090 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:11.742561 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:11.742472 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerStarted","Data":"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a"} Jul 15 08:40:12.338080 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:12.338050 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:12.339161 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:12.339134 2581 patch_prober.go:28] interesting pod/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 08:40:12.339291 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:12.339191 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 08:40:13.338910 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:13.338873 2581 patch_prober.go:28] interesting pod/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 08:40:13.339295 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:13.338959 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 08:40:14.338607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:14.338576 2581 patch_prober.go:28] interesting pod/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 08:40:14.338797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:14.338624 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 08:40:15.338779 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:15.338746 2581 patch_prober.go:28] interesting pod/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 15 08:40:15.339177 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:15.338811 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 15 08:40:16.342485 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:16.342458 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:16.342886 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:16.342715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:16.343311 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:16.343293 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:40:36.641795 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:36.641763 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:40:36.642299 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:36.642019 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" containerID="cri-o://896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86" gracePeriod=30 Jul 15 08:40:37.246340 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:37.246312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:40:37.252994 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:37.252974 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:40:38.016448 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:38.016414 2581 patch_prober.go:28] interesting pod/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:40:38.016815 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:38.016485 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:40:41.882034 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.882010 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:41.892178 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.892153 2581 generic.go:356] "Generic (PLEG): container finished" podID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerID="896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86" exitCode=0 Jul 15 08:40:41.892277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.892200 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" Jul 15 08:40:41.892277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.892236 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerDied","Data":"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86"} Jul 15 08:40:41.892385 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.892292 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns" event={"ID":"afbc9730-5907-478b-b2fc-5085a783ffaa","Type":"ContainerDied","Data":"896bb9e5f8a0ba6731b45ababeeb77b366024496c5805b5c73cc00def0c3d824"} Jul 15 08:40:41.892385 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.892324 2581 scope.go:117] "RemoveContainer" containerID="896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86" Jul 15 08:40:41.900797 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.900780 2581 scope.go:117] "RemoveContainer" containerID="2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235" Jul 15 08:40:41.910778 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.910751 2581 scope.go:117] "RemoveContainer" containerID="896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86" Jul 15 08:40:41.911044 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:40:41.911026 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86\": container with ID starting with 896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86 not found: ID does not exist" containerID="896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86" Jul 15 08:40:41.911095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.911052 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86"} err="failed to get container status \"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86\": rpc error: code = NotFound desc = could not find container \"896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86\": container with ID starting with 896bbbf24befc9678f5d279391175d8ad8f40859272548d9e05fe6f2cc8a0b86 not found: ID does not exist" Jul 15 08:40:41.911095 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.911069 2581 scope.go:117] "RemoveContainer" containerID="2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235" Jul 15 08:40:41.911310 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:40:41.911291 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235\": container with ID starting with 2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235 not found: ID does not exist" containerID="2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235" Jul 15 08:40:41.911369 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:41.911315 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235"} err="failed to get container status \"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235\": rpc error: code = NotFound desc = could not find container \"2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235\": container with ID starting with 2bf5f48e0849cdb64c403bd79b032b2de4956bb6b9cbe27def2d32f6ff6b1235 not found: ID does not exist" Jul 15 08:40:42.006218 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006193 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006335 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006223 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006335 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006298 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006450 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006339 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzd5m\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006450 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006396 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006450 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006428 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006464 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006494 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006605 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006521 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket\") pod \"afbc9730-5907-478b-b2fc-5085a783ffaa\" (UID: \"afbc9730-5907-478b-b2fc-5085a783ffaa\") " Jul 15 08:40:42.006780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006617 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:40:42.006872 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006844 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:40:42.006944 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006888 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.007004 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.006974 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:40:42.007128 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.007106 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:40:42.007372 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.007328 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:40:42.008528 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.008503 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:40:42.008628 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.008586 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:40:42.008690 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.008645 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:40:42.008746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.008730 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m" (OuterVolumeSpecName: "kube-api-access-vzd5m") pod "afbc9730-5907-478b-b2fc-5085a783ffaa" (UID: "afbc9730-5907-478b-b2fc-5085a783ffaa"). InnerVolumeSpecName "kube-api-access-vzd5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:40:42.107771 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107742 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107771 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107766 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107779 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzd5m\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-kube-api-access-vzd5m\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107793 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107804 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbc9730-5907-478b-b2fc-5085a783ffaa-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107817 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107829 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbc9730-5907-478b-b2fc-5085a783ffaa-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.107942 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.107842 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbc9730-5907-478b-b2fc-5085a783ffaa-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:40:42.214609 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.214583 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:40:42.219422 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:42.219402 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ec03f1-openshift-default-6d55c877d7-4njns"] Jul 15 08:40:43.351748 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:40:43.351715 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" path="/var/lib/kubelet/pods/afbc9730-5907-478b-b2fc-5085a783ffaa/volumes" Jul 15 08:41:02.966979 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:02.966898 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:41:02.967531 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:02.967180 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" containerID="cri-o://4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a" gracePeriod=30 Jul 15 08:41:08.215008 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.214985 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:41:08.233622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233577 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233622 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233616 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233723 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233639 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233723 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233673 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233723 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233716 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233868 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233752 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233868 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233868 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkfz4\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.233868 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.233834 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs\") pod \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\" (UID: \"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4\") " Jul 15 08:41:08.234189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.234077 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:41:08.234189 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.234116 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:41:08.234331 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.234304 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:41:08.234331 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.234314 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:41:08.234448 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.234420 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:41:08.235842 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.235813 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:41:08.236044 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.236018 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:41:08.236206 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.236182 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:41:08.236288 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.236234 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4" (OuterVolumeSpecName: "kube-api-access-gkfz4") pod "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" (UID: "9b5938f3-8831-43de-ba53-d8ebc1e7cbc4"). InnerVolumeSpecName "kube-api-access-gkfz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:41:08.334467 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334435 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334467 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334461 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334467 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334470 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334478 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334487 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334495 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334503 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334512 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:08.334704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:08.334521 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkfz4\" (UniqueName: \"kubernetes.io/projected/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4-kube-api-access-gkfz4\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:41:09.025277 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.025241 2581 generic.go:356] "Generic (PLEG): container finished" podID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerID="4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a" exitCode=0 Jul 15 08:41:09.025457 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.025292 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" Jul 15 08:41:09.025457 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.025308 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerDied","Data":"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a"} Jul 15 08:41:09.025457 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.025378 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d" event={"ID":"9b5938f3-8831-43de-ba53-d8ebc1e7cbc4","Type":"ContainerDied","Data":"51c5181e0592d16a123ed45a2433a24308895d534125862e96a502aed34447ac"} Jul 15 08:41:09.025457 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.025410 2581 scope.go:117] "RemoveContainer" containerID="4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a" Jul 15 08:41:09.034353 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.034336 2581 scope.go:117] "RemoveContainer" containerID="a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b" Jul 15 08:41:09.041936 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.041917 2581 scope.go:117] "RemoveContainer" containerID="4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a" Jul 15 08:41:09.042166 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:41:09.042147 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a\": container with ID starting with 4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a not found: ID does not exist" containerID="4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a" Jul 15 08:41:09.042212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.042174 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a"} err="failed to get container status \"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a\": rpc error: code = NotFound desc = could not find container \"4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a\": container with ID starting with 4ac26b59cac77f3c6e2f922ebde3669d5e2ab22822de7e8d86e82c411bf2932a not found: ID does not exist" Jul 15 08:41:09.042212 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.042191 2581 scope.go:117] "RemoveContainer" containerID="a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b" Jul 15 08:41:09.042437 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:41:09.042416 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b\": container with ID starting with a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b not found: ID does not exist" containerID="a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b" Jul 15 08:41:09.042536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.042442 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b"} err="failed to get container status \"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b\": rpc error: code = NotFound desc = could not find container \"a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b\": container with ID starting with a5664d7743852f4fa6859379129a0cffaa390ee065fc2aed2b308d961694924b not found: ID does not exist" Jul 15 08:41:09.048060 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.048037 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:41:09.052399 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.052382 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7a0e41-openshift-default-6d7bddbcf7-hp92d"] Jul 15 08:41:09.351740 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.351714 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" path="/var/lib/kubelet/pods/9b5938f3-8831-43de-ba53-d8ebc1e7cbc4/volumes" Jul 15 08:41:09.790318 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790232 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:41:09.790613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790599 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.790613 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790614 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790626 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790632 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790646 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790662 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790673 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790678 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790732 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790740 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.790746 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790748 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5938f3-8831-43de-ba53-d8ebc1e7cbc4" containerName="istio-proxy" Jul 15 08:41:09.791209 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.790754 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="afbc9730-5907-478b-b2fc-5085a783ffaa" containerName="istio-proxy" Jul 15 08:41:09.793608 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.793584 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.796402 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.796380 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 08:41:09.796511 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.796380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8e59d0-serving-cert\"" Jul 15 08:41:09.797209 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.797190 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 08:41:09.797479 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.797462 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cd6kw\"" Jul 15 08:41:09.806915 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.806885 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:41:09.846072 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.846046 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkgjz\" (UniqueName: \"kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.846201 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.846095 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.846201 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.846120 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.947218 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.947186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.947418 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.947222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.947418 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.947300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkgjz\" (UniqueName: \"kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.947418 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:41:09.947388 2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-serving-cert: secret "maas-api-e2e-shared-a-8e59d0-serving-cert" not found Jul 15 08:41:09.947616 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:41:09.947466 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls podName:703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f nodeName:}" failed. No retries permitted until 2026-07-15 08:41:10.447450009 +0000 UTC m=+2433.838854609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls") pod "maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" (UID: "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f") : secret "maas-api-e2e-shared-a-8e59d0-serving-cert" not found Jul 15 08:41:09.947989 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.947966 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:09.956283 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:09.956260 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkgjz\" (UniqueName: \"kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:10.451323 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:10.451286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:10.453690 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:10.453671 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") pod \"maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:10.706818 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:10.706741 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:10.843577 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:10.843553 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:41:10.844800 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:41:10.844775 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703d4f6a_ec0f_4f72_9c5e_5f29d43dc91f.slice/crio-f4d84198cc98f4aca5b7aec3d141479f4e5dc0227b96c8f272b1157f8ac16327 WatchSource:0}: Error finding container f4d84198cc98f4aca5b7aec3d141479f4e5dc0227b96c8f272b1157f8ac16327: Status 404 returned error can't find the container with id f4d84198cc98f4aca5b7aec3d141479f4e5dc0227b96c8f272b1157f8ac16327 Jul 15 08:41:10.846230 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:10.846209 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 08:41:11.043081 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:11.042996 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" event={"ID":"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f","Type":"ContainerStarted","Data":"f4d84198cc98f4aca5b7aec3d141479f4e5dc0227b96c8f272b1157f8ac16327"} Jul 15 08:41:14.059690 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.059650 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" event={"ID":"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f","Type":"ContainerStarted","Data":"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b"} Jul 15 08:41:14.060144 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.059735 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:14.078532 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.078487 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" podStartSLOduration=2.893236578 podStartE2EDuration="5.078471747s" totalImagesPullingTime="2.185235169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:41:14.076978816 +0000 UTC m=+2437.468383440" watchObservedRunningTime="2026-07-15 08:41:14.078471747 +0000 UTC m=+2437.469876370" Jul 15 08:41:14.935887 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.935858 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:41:14.939267 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.939248 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.941812 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.941792 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1db148-openshift-default-dockercfg-65vm7\"" Jul 15 08:41:14.949655 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.949632 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:41:14.994616 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994587 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.994780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994622 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.994780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.994780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.994780 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994769 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwrdj\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.995022 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.995022 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.995022 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:14.995022 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:14.994962 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.095995 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.095964 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096115 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096476 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096403 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096533 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096566 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwrdj\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.096761 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.097074 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.096995 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.097296 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.097275 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.098516 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.098495 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.098930 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.098912 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.107255 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.107232 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.107881 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.107864 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwrdj\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj\") pod \"e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.250284 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.250206 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:15.390751 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.390664 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:41:15.392843 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:41:15.392819 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded20bad_721d_4cff_8336_9f1b3921f362.slice/crio-daeb5aadcd70ace00cc939b19352faec1b4660cde3900b049130917be729b5bb WatchSource:0}: Error finding container daeb5aadcd70ace00cc939b19352faec1b4660cde3900b049130917be729b5bb: Status 404 returned error can't find the container with id daeb5aadcd70ace00cc939b19352faec1b4660cde3900b049130917be729b5bb Jul 15 08:41:15.395053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.395024 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:15.395121 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.395086 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:15.395121 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:15.395113 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:16.071861 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:16.071826 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerStarted","Data":"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77"} Jul 15 08:41:16.072072 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:16.071883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerStarted","Data":"daeb5aadcd70ace00cc939b19352faec1b4660cde3900b049130917be729b5bb"} Jul 15 08:41:16.095725 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:16.095680 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podStartSLOduration=2.095664735 podStartE2EDuration="2.095664735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 08:41:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:41:16.093482901 +0000 UTC m=+2439.484887544" watchObservedRunningTime="2026-07-15 08:41:16.095664735 +0000 UTC m=+2439.487069357" Jul 15 08:41:16.251316 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:16.251282 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:16.256040 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:16.256017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:17.077376 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:17.077323 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:17.078182 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:17.078163 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:20.070563 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:20.070535 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:41:32.078847 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.078747 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 15 08:41:32.078847 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.078814 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 15 08:41:32.152732 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.152701 2581 generic.go:356] "Generic (PLEG): container finished" podID="ded20bad-721d-4cff-8336-9f1b3921f362" containerID="d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77" exitCode=0 Jul 15 08:41:32.152891 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.152747 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerDied","Data":"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77"} Jul 15 08:41:32.153108 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.153085 2581 scope.go:117] "RemoveContainer" containerID="d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77" Jul 15 08:41:32.154096 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.154057 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:32.154174 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.154155 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:32.154220 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:32.154199 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 15 08:41:33.159434 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:33.159394 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerStarted","Data":"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa"} Jul 15 08:41:33.251426 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:33.251392 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:33.251604 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:33.251590 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 15 08:41:33.251654 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:33.251635 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 15 08:41:34.251681 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:34.251649 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 15 08:41:34.252085 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:34.251703 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 15 08:41:35.251480 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:35.251445 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 15 08:41:35.251649 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:35.251497 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 15 08:41:36.250838 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:36.250805 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 15 08:41:36.251232 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:36.250872 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 15 08:41:37.254304 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:37.254270 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:37.254764 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:37.254548 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:41:37.255179 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:41:37.255162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:45:37.280795 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:45:37.280722 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:45:37.285082 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:45:37.285055 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-htgqh_d6a9131f-37bd-480d-aa96-b226cef6d2c1/ovn-acl-logging/0.log" Jul 15 08:48:23.471527 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.471496 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:48:23.472056 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.471717 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" podUID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" containerName="maas-api" containerID="cri-o://4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b" gracePeriod=30 Jul 15 08:48:23.730037 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.729973 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:48:23.768984 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.768952 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") pod \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " Jul 15 08:48:23.769142 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.769033 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca\") pod \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " Jul 15 08:48:23.769142 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.769067 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkgjz\" (UniqueName: \"kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz\") pod \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\" (UID: \"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f\") " Jul 15 08:48:23.769618 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.769588 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" (UID: "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:48:23.771167 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.771140 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" (UID: "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 08:48:23.771600 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.771580 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz" (OuterVolumeSpecName: "kube-api-access-fkgjz") pod "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" (UID: "703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f"). InnerVolumeSpecName "kube-api-access-fkgjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:48:23.870100 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.870079 2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-maas-api-tls\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:48:23.870100 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.870101 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-openshift-service-ca\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:48:23.870252 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:23.870112 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkgjz\" (UniqueName: \"kubernetes.io/projected/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f-kube-api-access-fkgjz\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:48:24.145066 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.145032 2581 generic.go:356] "Generic (PLEG): container finished" podID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" containerID="4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b" exitCode=0 Jul 15 08:48:24.145263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.145078 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" Jul 15 08:48:24.145263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.145114 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" event={"ID":"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f","Type":"ContainerDied","Data":"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b"} Jul 15 08:48:24.145263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.145166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc" event={"ID":"703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f","Type":"ContainerDied","Data":"f4d84198cc98f4aca5b7aec3d141479f4e5dc0227b96c8f272b1157f8ac16327"} Jul 15 08:48:24.145263 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.145190 2581 scope.go:117] "RemoveContainer" containerID="4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b" Jul 15 08:48:24.154741 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.154718 2581 scope.go:117] "RemoveContainer" containerID="4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b" Jul 15 08:48:24.154974 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:48:24.154958 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b\": container with ID starting with 4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b not found: ID does not exist" containerID="4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b" Jul 15 08:48:24.155025 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.154981 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b"} err="failed to get container status \"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b\": rpc error: code = NotFound desc = could not find container \"4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b\": container with ID starting with 4668514d12f579715357db9142ddfff6ce0336d2851b0112b9fd3a556b4f118b not found: ID does not exist" Jul 15 08:48:24.168627 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.168600 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:48:24.173045 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:24.173024 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8e59d0-646fcb5955-k4qpc"] Jul 15 08:48:25.352651 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:48:25.352620 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" path="/var/lib/kubelet/pods/703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f/volumes" Jul 15 08:49:05.047873 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:05.047838 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:49:05.048511 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:05.048085 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" containerID="cri-o://db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa" gracePeriod=30 Jul 15 08:49:05.871561 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:05.871533 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db69d568c-mspsf_95967b37-180e-41f7-bc94-de3cf05bae03/authorino/0.log" Jul 15 08:49:07.257160 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:07.257125 2581 patch_prober.go:28] interesting pod/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 08:49:07.257558 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:07.257181 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 08:49:10.318440 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.318388 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:49:10.366541 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.366510 2581 generic.go:356] "Generic (PLEG): container finished" podID="ded20bad-721d-4cff-8336-9f1b3921f362" containerID="db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa" exitCode=0 Jul 15 08:49:10.366731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.366557 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" Jul 15 08:49:10.366731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.366603 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerDied","Data":"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa"} Jul 15 08:49:10.366731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.366660 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx" event={"ID":"ded20bad-721d-4cff-8336-9f1b3921f362","Type":"ContainerDied","Data":"daeb5aadcd70ace00cc939b19352faec1b4660cde3900b049130917be729b5bb"} Jul 15 08:49:10.366731 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.366704 2581 scope.go:117] "RemoveContainer" containerID="db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa" Jul 15 08:49:10.375039 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.375024 2581 scope.go:117] "RemoveContainer" containerID="d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77" Jul 15 08:49:10.382405 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.382387 2581 scope.go:117] "RemoveContainer" containerID="db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa" Jul 15 08:49:10.382694 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:49:10.382675 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa\": container with ID starting with db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa not found: ID does not exist" containerID="db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa" Jul 15 08:49:10.382743 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.382701 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa"} err="failed to get container status \"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa\": rpc error: code = NotFound desc = could not find container \"db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa\": container with ID starting with db10bee77c3c0ca4baa7289c138d2375b59a7d3cd4e2fe0eb77a754cae857eaa not found: ID does not exist" Jul 15 08:49:10.382743 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.382718 2581 scope.go:117] "RemoveContainer" containerID="d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77" Jul 15 08:49:10.382967 ip-10-0-141-240 kubenswrapper[2581]: E0715 08:49:10.382949 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77\": container with ID starting with d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77 not found: ID does not exist" containerID="d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77" Jul 15 08:49:10.383016 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.382974 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77"} err="failed to get container status \"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77\": rpc error: code = NotFound desc = could not find container \"d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77\": container with ID starting with d046fe5c9b4bae5249348fbe3909154c9c1ca7276ddc1a5cdecd104d11421a77 not found: ID does not exist" Jul 15 08:49:10.465689 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465663 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.465805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465700 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.465805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465721 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.465805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465743 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.465805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465767 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwrdj\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.465805 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.466062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465819 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.466062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465940 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:49:10.466062 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465972 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:49:10.466222 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466155 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 08:49:10.466222 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466214 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:49:10.466328 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.465877 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.466328 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466267 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo\") pod \"ded20bad-721d-4cff-8336-9f1b3921f362\" (UID: \"ded20bad-721d-4cff-8336-9f1b3921f362\") " Jul 15 08:49:10.466660 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466636 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.466660 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466660 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-credential-socket\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.466821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466682 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ded20bad-721d-4cff-8336-9f1b3921f362-istiod-ca-cert\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.466821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466695 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-workload-certs\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.466821 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.466750 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data" (OuterVolumeSpecName: "istio-data") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:49:10.468070 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.468046 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token" (OuterVolumeSpecName: "istio-token") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:49:10.468221 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.468196 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj" (OuterVolumeSpecName: "kube-api-access-pwrdj") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "kube-api-access-pwrdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 08:49:10.468384 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.468339 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 08:49:10.468502 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.468451 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ded20bad-721d-4cff-8336-9f1b3921f362" (UID: "ded20bad-721d-4cff-8336-9f1b3921f362"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 08:49:10.567698 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.567675 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-envoy\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.567698 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.567696 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwrdj\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-kube-api-access-pwrdj\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.567849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.567708 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ded20bad-721d-4cff-8336-9f1b3921f362-istio-data\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.567849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.567716 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded20bad-721d-4cff-8336-9f1b3921f362-istio-token\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.567849 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.567723 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ded20bad-721d-4cff-8336-9f1b3921f362-istio-podinfo\") on node \"ip-10-0-141-240.ec2.internal\" DevicePath \"\"" Jul 15 08:49:10.688722 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.688697 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:49:10.692928 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:10.692907 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1db148-openshift-default-6b84b6f8d7-qhfdx"] Jul 15 08:49:11.074679 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.074587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wlm7l_29bff1dd-e66f-4564-8e95-8a09fd61e029/manager/0.log" Jul 15 08:49:11.211199 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.211167 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kzvf4_aca98520-8c57-4354-a1e5-c59716fbd813/manager/0.log" Jul 15 08:49:11.352844 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.352813 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" path="/var/lib/kubelet/pods/ded20bad-721d-4cff-8336-9f1b3921f362/volumes" Jul 15 08:49:11.473053 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.473027 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jbwq8_b98ee470-8fb3-47a7-bfc2-ec216305b98f/server/0.log" Jul 15 08:49:11.605795 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.605694 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-25n98_65403957-a5d6-48c7-aa23-f2a8fb8b1319/manager/0.log" Jul 15 08:49:11.863074 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.862990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b8c85cff6-k9p7d_ace6dbfa-cf66-47db-916e-bba2a68e9955/copy-manifests/0.log" Jul 15 08:49:11.869973 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:11.869947 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b8c85cff6-k9p7d_ace6dbfa-cf66-47db-916e-bba2a68e9955/manager/0.log" Jul 15 08:49:15.732244 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:15.732216 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db69d568c-mspsf_95967b37-180e-41f7-bc94-de3cf05bae03/authorino/0.log" Jul 15 08:49:15.869823 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:15.869796 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8qg54_12a196c1-1cad-43ee-8bc6-49088a1339bd/manager/0.log" Jul 15 08:49:15.984740 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:15.984613 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4cwxh_a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a/manager/0.log" Jul 15 08:49:16.231607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:16.231579 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pcl55_f932eb95-29ec-498a-89d8-8c4d9f9d3066/registry-server/0.log" Jul 15 08:49:16.366489 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:16.366464 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kwtqn_6c87db7b-48e7-4b21-947d-ba13a4b05dba/manager/0.log" Jul 15 08:49:16.632795 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:16.632713 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kmbwz_7e440794-5f15-40bf-9288-712c16bd41fa/manager/0.log" Jul 15 08:49:17.011570 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:17.011490 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gdmx6_c6929599-b919-4016-ab37-82dd9d3604c3/istio-proxy/0.log" Jul 15 08:49:17.270748 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:17.270660 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-657b79bd8d-9dqm7_49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2/kube-auth-proxy/0.log" Jul 15 08:49:17.525130 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:17.525040 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/1.log" Jul 15 08:49:18.258696 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:18.258658 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-sdvh9_70fa483a-76ba-4c08-8736-6ef8aa73b530/main/0.log" Jul 15 08:49:18.613951 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:18.613908 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5jkns_4ec3ad3c-e078-436a-a73f-06afc05a3a23/main/0.log" Jul 15 08:49:26.040515 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:26.040481 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c2847_f6dd6fb1-a94d-4351-b84a-8550c44770d4/global-pull-secret-syncer/0.log" Jul 15 08:49:26.228931 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:26.228900 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lcvbr_5bf7d91e-5e46-4b77-8d49-1abf0ec5348b/konnectivity-agent/0.log" Jul 15 08:49:26.341719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:26.341686 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-240.ec2.internal_869786661d939b62b49e72c2e18cdc59/haproxy/0.log" Jul 15 08:49:31.187641 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.187601 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6db69d568c-mspsf_95967b37-180e-41f7-bc94-de3cf05bae03/authorino/0.log" Jul 15 08:49:31.226411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.226348 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8qg54_12a196c1-1cad-43ee-8bc6-49088a1339bd/manager/0.log" Jul 15 08:49:31.259217 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.259179 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4cwxh_a45e74a8-6f65-4d0c-9baf-9fba01f7ae0a/manager/0.log" Jul 15 08:49:31.341597 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.341558 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pcl55_f932eb95-29ec-498a-89d8-8c4d9f9d3066/registry-server/0.log" Jul 15 08:49:31.509201 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.509109 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kwtqn_6c87db7b-48e7-4b21-947d-ba13a4b05dba/manager/0.log" Jul 15 08:49:31.628912 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:31.628875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kmbwz_7e440794-5f15-40bf-9288-712c16bd41fa/manager/0.log" Jul 15 08:49:33.948960 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:33.948935 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9m5jt_9e607e94-651c-479b-80e1-ee44caa4177f/node-exporter/0.log" Jul 15 08:49:33.975262 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:33.975236 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9m5jt_9e607e94-651c-479b-80e1-ee44caa4177f/kube-rbac-proxy/0.log" Jul 15 08:49:33.997712 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:33.997690 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9m5jt_9e607e94-651c-479b-80e1-ee44caa4177f/init-textfile/0.log" Jul 15 08:49:34.379280 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:34.379254 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cq42h_183a9041-0728-4f42-b9ce-a608eed01d56/prometheus-operator/0.log" Jul 15 08:49:34.397652 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:34.397629 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-cq42h_183a9041-0728-4f42-b9ce-a608eed01d56/kube-rbac-proxy/0.log" Jul 15 08:49:35.594011 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.593979 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964"] Jul 15 08:49:35.594411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594368 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594380 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594399 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594405 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594411 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594413 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" containerName="maas-api" Jul 15 08:49:35.594591 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594419 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" containerName="maas-api" Jul 15 08:49:35.594591 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594479 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594591 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594488 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded20bad-721d-4cff-8336-9f1b3921f362" containerName="istio-proxy" Jul 15 08:49:35.594591 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.594494 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="703d4f6a-ec0f-4f72-9c5e-5f29d43dc91f" containerName="maas-api" Jul 15 08:49:35.597536 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.597512 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.600268 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.600246 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bt6dg\"/\"kube-root-ca.crt\"" Jul 15 08:49:35.600393 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.600252 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bt6dg\"/\"default-dockercfg-nbpnw\"" Jul 15 08:49:35.601173 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.601160 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bt6dg\"/\"openshift-service-ca.crt\"" Jul 15 08:49:35.608338 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.608318 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964"] Jul 15 08:49:35.670079 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.670047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4zv7\" (UniqueName: \"kubernetes.io/projected/763768d9-43ed-4ac4-b42b-c038f84b7d69-kube-api-access-p4zv7\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.670079 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.670084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-podres\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.670322 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.670184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-sys\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.670322 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.670221 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-lib-modules\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.670322 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.670270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-proc\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771433 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-sys\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771433 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-lib-modules\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-proc\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4zv7\" (UniqueName: \"kubernetes.io/projected/763768d9-43ed-4ac4-b42b-c038f84b7d69-kube-api-access-p4zv7\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771517 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-sys\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-podres\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-podres\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771614 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-proc\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.771669 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.771603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/763768d9-43ed-4ac4-b42b-c038f84b7d69-lib-modules\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.780983 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.780959 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4zv7\" (UniqueName: \"kubernetes.io/projected/763768d9-43ed-4ac4-b42b-c038f84b7d69-kube-api-access-p4zv7\") pod \"perf-node-gather-daemonset-45964\" (UID: \"763768d9-43ed-4ac4-b42b-c038f84b7d69\") " pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:35.854151 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.854082 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-plj4m_f292e350-c244-4f24-bcf8-b25d19bbe904/networking-console-plugin/0.log" Jul 15 08:49:35.907173 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:35.907145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:36.040102 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:36.040078 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964"] Jul 15 08:49:36.042698 ip-10-0-141-240 kubenswrapper[2581]: W0715 08:49:36.042655 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod763768d9_43ed_4ac4_b42b_c038f84b7d69.slice/crio-0ddae67f94834e5f1bb1130e397dda64600f394c5d8654908e8470def247d484 WatchSource:0}: Error finding container 0ddae67f94834e5f1bb1130e397dda64600f394c5d8654908e8470def247d484: Status 404 returned error can't find the container with id 0ddae67f94834e5f1bb1130e397dda64600f394c5d8654908e8470def247d484 Jul 15 08:49:36.493533 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:36.493446 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" event={"ID":"763768d9-43ed-4ac4-b42b-c038f84b7d69","Type":"ContainerStarted","Data":"542f52caea25620e4a4248277502f963a90b20a6f764e25184357f60fe5aac69"} Jul 15 08:49:36.493533 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:36.493504 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" event={"ID":"763768d9-43ed-4ac4-b42b-c038f84b7d69","Type":"ContainerStarted","Data":"0ddae67f94834e5f1bb1130e397dda64600f394c5d8654908e8470def247d484"} Jul 15 08:49:36.493719 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:36.493582 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:36.511377 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:36.511316 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" podStartSLOduration=1.511304185 podStartE2EDuration="1.511304185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 08:49:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 08:49:36.510104951 +0000 UTC m=+2939.901509575" watchObservedRunningTime="2026-07-15 08:49:36.511304185 +0000 UTC m=+2939.902708807" Jul 15 08:49:37.020088 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:37.020045 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6df465f9dd-gxsk8_233afcd7-5d37-4d8a-a657-5e96b12150b9/console/0.log" Jul 15 08:49:38.800568 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:38.800530 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xbhjm_efb6b8a5-1075-4078-a7ce-ded389856f53/dns/0.log" Jul 15 08:49:38.823015 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:38.822980 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xbhjm_efb6b8a5-1075-4078-a7ce-ded389856f53/kube-rbac-proxy/0.log" Jul 15 08:49:38.901975 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:38.901952 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtzzx_bd80e0b2-f15c-41fa-96da-73cf4aba4e81/dns-node-resolver/0.log" Jul 15 08:49:39.563589 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:39.563563 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zcxrr_7ddf0841-975e-4676-a0c1-a7133fa9549d/node-ca/0.log" Jul 15 08:49:40.530709 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:40.530680 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gdmx6_c6929599-b919-4016-ab37-82dd9d3604c3/istio-proxy/0.log" Jul 15 08:49:40.667607 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:40.667577 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-657b79bd8d-9dqm7_49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2/kube-auth-proxy/0.log" Jul 15 08:49:40.727084 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:40.727059 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/0.log" Jul 15 08:49:40.784494 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:40.784417 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/1.log" Jul 15 08:49:41.631301 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:41.631277 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qxb7m_6733015d-b01c-482a-b99e-5891d32f5a9f/serve-healthcheck-canary/0.log" Jul 15 08:49:42.295032 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:42.294998 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgxmz_359e98f9-0feb-491c-bbc2-14f0d058d07c/kube-rbac-proxy/0.log" Jul 15 08:49:42.315665 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:42.315641 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgxmz_359e98f9-0feb-491c-bbc2-14f0d058d07c/exporter/0.log" Jul 15 08:49:42.338828 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:42.338808 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgxmz_359e98f9-0feb-491c-bbc2-14f0d058d07c/extractor/0.log" Jul 15 08:49:42.507100 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:42.507076 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bt6dg/perf-node-gather-daemonset-45964" Jul 15 08:49:44.632114 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:44.632085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wlm7l_29bff1dd-e66f-4564-8e95-8a09fd61e029/manager/0.log" Jul 15 08:49:44.858163 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:44.858134 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kzvf4_aca98520-8c57-4354-a1e5-c59716fbd813/manager/0.log" Jul 15 08:49:44.980704 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:44.980685 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jbwq8_b98ee470-8fb3-47a7-bfc2-ec216305b98f/server/0.log" Jul 15 08:49:45.086485 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:45.086454 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-25n98_65403957-a5d6-48c7-aa23-f2a8fb8b1319/manager/0.log" Jul 15 08:49:45.170522 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:45.170495 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b8c85cff6-k9p7d_ace6dbfa-cf66-47db-916e-bba2a68e9955/manager/0.log" Jul 15 08:49:45.196573 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:45.196553 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b8c85cff6-k9p7d_ace6dbfa-cf66-47db-916e-bba2a68e9955/copy-manifests/0.log" Jul 15 08:49:46.587089 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:46.587034 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cvstv_3ccae9cf-5aa7-42ad-82ff-e6652e4519e2/openshift-lws-operator/0.log" Jul 15 08:49:46.658969 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:46.658943 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gdmx6_c6929599-b919-4016-ab37-82dd9d3604c3/istio-proxy/0.log" Jul 15 08:49:46.784672 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:46.784628 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-657b79bd8d-9dqm7_49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2/kube-auth-proxy/0.log" Jul 15 08:49:46.825660 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:46.825579 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/0.log" Jul 15 08:49:46.885529 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:46.885506 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/1.log" Jul 15 08:49:48.116680 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:48.116649 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gdmx6_c6929599-b919-4016-ab37-82dd9d3604c3/istio-proxy/0.log" Jul 15 08:49:48.231176 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:48.231142 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-657b79bd8d-9dqm7_49e34b7e-2ad0-4fa3-9e1f-abc59c119fe2/kube-auth-proxy/0.log" Jul 15 08:49:48.280520 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:48.280500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/0.log" Jul 15 08:49:48.336530 ip-10-0-141-240 kubenswrapper[2581]: I0715 08:49:48.336503 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-75t2p_0cfe7135-0067-4a8d-81b0-3cd0817de42d/istio-proxy/1.log"