Jul 23 09:14:51.193732 ip-10-0-132-192 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 09:14:51.193741 ip-10-0-132-192 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 09:14:51.193748 ip-10-0-132-192 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 09:14:51.193959 ip-10-0-132-192 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 09:15:01.252764 ip-10-0-132-192 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 09:15:01.252782 ip-10-0-132-192 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 7995ad3c304b4c84b85e577c5f031405 -- Jul 23 09:16:59.677634 ip-10-0-132-192 systemd[1]: Starting Kubernetes Kubelet... Jul 23 09:17:00.066779 ip-10-0-132-192 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:17:00.066779 ip-10-0-132-192 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 09:17:00.066779 ip-10-0-132-192 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:17:00.066779 ip-10-0-132-192 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 09:17:00.066779 ip-10-0-132-192 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 09:17:00.067965 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.067870 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073933 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073952 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073957 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073961 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073965 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.073959 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073969 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073973 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073977 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073981 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073985 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073988 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073992 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073995 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.073999 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074003 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074006 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074010 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074014 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074017 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074021 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074026 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074030 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074034 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074038 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074042 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.074298 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074046 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074050 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074054 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074057 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074065 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074070 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074073 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074079 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074084 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074089 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074093 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074097 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074102 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074107 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074113 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074119 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074125 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074129 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074134 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.075215 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074138 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074143 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074148 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074153 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074157 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074161 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074165 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074170 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074174 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074178 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074183 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074187 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074192 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074196 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074201 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074205 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074209 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074214 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074218 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074222 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.076080 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074226 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074230 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074234 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074240 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074244 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074248 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074253 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074258 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074262 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074267 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074271 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074275 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074280 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074284 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074288 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074295 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074301 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074306 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074310 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074315 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.076919 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074319 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074323 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074346 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074350 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074354 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074358 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074362 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074366 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074371 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074375 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074381 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074386 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074389 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074394 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.074398 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075606 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075618 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075622 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075626 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.077793 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075632 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075640 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075647 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075652 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075657 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075662 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075667 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075672 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075677 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075682 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075688 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075692 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075696 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075701 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075705 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075710 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075714 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075719 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075723 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.078632 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075727 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075732 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075736 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075740 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075744 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075749 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075753 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075757 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075762 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075768 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075772 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075780 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075785 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075790 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075795 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075800 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075804 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075809 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075814 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.079266 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075818 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075822 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075827 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075831 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075835 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075840 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075846 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075851 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075855 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075860 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075864 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075869 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075873 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075877 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075881 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075885 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075889 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075894 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075897 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.079966 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075902 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075906 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075913 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075917 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075921 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075925 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075929 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075934 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075938 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075942 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075946 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075950 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075954 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075959 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075963 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075967 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075972 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075976 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075980 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075985 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.080507 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075990 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075994 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.075998 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076002 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076006 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076010 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076014 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076019 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076023 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076027 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076031 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076035 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076039 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076043 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076048 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076052 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076057 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076061 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076733 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076741 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.081006 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076746 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076750 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076755 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076762 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076767 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076772 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076777 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076785 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076789 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076793 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076798 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076803 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076807 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076812 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076816 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076820 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076824 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076828 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076832 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.081614 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076837 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076842 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076848 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076855 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076860 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076865 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076870 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076874 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076878 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076886 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076891 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076896 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076901 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076905 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076910 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076914 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076917 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076921 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076925 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.082094 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076929 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076937 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076942 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076946 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076950 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076955 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076960 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076966 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076970 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076975 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076979 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076983 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076987 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076991 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.076995 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077000 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077004 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077008 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077013 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.082589 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077017 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077021 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077025 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077032 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077036 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077040 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077044 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077048 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077052 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077056 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077060 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077064 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077069 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077073 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077078 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077083 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077087 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077091 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077096 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077100 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.083055 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077104 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077109 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077113 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077117 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077121 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077126 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077130 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077134 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077138 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077142 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077146 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077150 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077155 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077158 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077162 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077168 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077173 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077177 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077181 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.077186 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077307 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 09:17:00.083545 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077323 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077435 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077443 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077451 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077456 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077466 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077473 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077478 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077483 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077489 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077494 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077499 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077505 2562 flags.go:64] FLAG: --cgroup-root="" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077509 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077514 2562 flags.go:64] FLAG: --client-ca-file="" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077519 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077524 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077532 2562 flags.go:64] FLAG: --cluster-domain="" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077536 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077541 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077547 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077552 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077559 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 09:17:00.084053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077564 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077569 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077574 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077581 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077586 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077590 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077595 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077601 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077608 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077613 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077618 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077622 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077627 2562 flags.go:64] FLAG: --enable-server="true" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077632 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077640 2562 flags.go:64] FLAG: --event-burst="100" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077645 2562 flags.go:64] FLAG: --event-qps="50" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077650 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077654 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077659 2562 flags.go:64] FLAG: --eviction-hard="" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077665 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077669 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077674 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077680 2562 flags.go:64] FLAG: --eviction-soft="" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077685 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077690 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 09:17:00.084681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077694 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077699 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077703 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077708 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077712 2562 flags.go:64] FLAG: --feature-gates="" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077719 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077724 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077729 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077734 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077739 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077746 2562 flags.go:64] FLAG: --help="false" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077750 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077755 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077760 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077766 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077771 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077776 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077781 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077786 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077790 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077794 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077801 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077806 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077810 2562 flags.go:64] FLAG: --kube-reserved="" Jul 23 09:17:00.085375 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077814 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077819 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077824 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077829 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077834 2562 flags.go:64] FLAG: --lock-file="" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077839 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077845 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077850 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077858 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077863 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077868 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077873 2562 flags.go:64] FLAG: --logging-format="text" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077878 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077883 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077887 2562 flags.go:64] FLAG: --manifest-url="" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077892 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077899 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077903 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077911 2562 flags.go:64] FLAG: --max-pods="110" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077915 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077920 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077925 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077929 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077934 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077939 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 09:17:00.085957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077943 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077955 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077960 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077965 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077970 2562 flags.go:64] FLAG: --pod-cidr="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077974 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077983 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077987 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077992 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.077996 2562 flags.go:64] FLAG: --port="10250" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078001 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078006 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b72aaf25d133505a" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078011 2562 flags.go:64] FLAG: --qos-reserved="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078016 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078021 2562 flags.go:64] FLAG: --register-node="true" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078025 2562 flags.go:64] FLAG: --register-with-taints="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078031 2562 flags.go:64] FLAG: --registry-burst="10" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078036 2562 flags.go:64] FLAG: --registry-qps="5" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078040 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078045 2562 flags.go:64] FLAG: --reserved-memory="" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078050 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078056 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078061 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078066 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078071 2562 flags.go:64] FLAG: --runonce="false" Jul 23 09:17:00.086612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078076 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078081 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078086 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078090 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078095 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078101 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078105 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078110 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078115 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078120 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078124 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078128 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078133 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078138 2562 flags.go:64] FLAG: --system-cgroups="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078142 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078152 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078156 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078161 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078169 2562 flags.go:64] FLAG: --tls-min-version="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078173 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078177 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078182 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078187 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078192 2562 flags.go:64] FLAG: --v="2" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078199 2562 flags.go:64] FLAG: --version="false" Jul 23 09:17:00.087217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078205 2562 flags.go:64] FLAG: --vmodule="" Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078211 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078217 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078423 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078431 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078436 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078441 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078448 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078453 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078458 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078463 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078467 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078472 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078476 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078480 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078484 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078488 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078492 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078496 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078500 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.087824 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078504 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078508 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078512 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078517 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078521 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078525 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078530 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078534 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078538 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078542 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078547 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078551 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078555 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078559 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078563 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078567 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078572 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078576 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078580 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078584 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.088301 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078588 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078593 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078597 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078601 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078605 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078609 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078613 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078617 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078622 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078626 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078629 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078633 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078637 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078641 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078645 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078649 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078653 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078657 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078661 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078665 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.088798 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078669 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078674 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078678 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078682 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078686 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078690 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078694 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078699 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078703 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078709 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078713 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078718 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078722 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078728 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078732 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078736 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078740 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078744 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078750 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.089357 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078756 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078760 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078765 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078770 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078774 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078779 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078783 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078787 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078791 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078795 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078799 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078803 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078808 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078812 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078816 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078820 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078826 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078832 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078836 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078841 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.089827 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078845 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078849 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.078855 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.078868 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.085814 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.085830 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.085849 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.085858 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085914 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085919 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085924 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085928 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085931 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085934 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.090344 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085937 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085940 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085942 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085945 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085947 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085950 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085952 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085955 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085958 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085960 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085963 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085965 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085970 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085973 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085976 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085979 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085982 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085985 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085988 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.090695 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085990 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085993 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085995 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.085998 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086002 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086005 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086007 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086010 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086012 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086015 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086017 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086020 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086022 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086025 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086027 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086030 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086032 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086034 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086037 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086039 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.091147 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086042 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086044 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086047 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086049 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086052 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086054 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086057 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086061 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086064 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086066 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086069 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086071 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086074 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086076 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086079 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086081 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086084 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086087 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086090 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.091648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086093 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086095 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086097 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086100 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086103 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086105 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086108 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086111 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086113 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086116 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086119 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086121 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086123 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086126 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086128 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086131 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086133 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086136 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086138 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086140 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.092112 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086143 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086145 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086148 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086150 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086153 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086155 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086158 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086161 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086163 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086166 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086169 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086171 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086174 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086176 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086179 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.086183 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:17:00.092608 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086318 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086324 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086347 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086351 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086355 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086358 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086361 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086364 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086366 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086369 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086372 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086375 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086377 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086380 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086382 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086385 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086389 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086393 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086396 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086399 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 23 09:17:00.093052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086401 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086404 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086406 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086409 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086411 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086414 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086416 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086418 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086421 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086424 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086427 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086429 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086432 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086434 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086437 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086439 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086441 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086444 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086446 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086449 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 09:17:00.093585 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086451 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086453 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086456 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086458 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086460 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086463 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086465 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086468 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086470 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086472 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086475 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086478 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086480 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086482 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086485 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086487 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086491 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086494 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086496 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 09:17:00.094052 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086499 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086502 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086505 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086507 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086510 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086512 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086514 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086517 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086519 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086522 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086524 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086526 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086529 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086531 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086534 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086536 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086539 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086542 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086545 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 09:17:00.094671 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086548 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086552 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086554 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086557 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086559 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086562 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086564 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086566 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086569 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086571 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086574 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086576 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086579 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086581 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086584 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086586 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086589 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086592 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086594 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086596 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 09:17:00.095438 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:00.086599 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.086603 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.086715 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.088742 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.090156 2562 server.go:1019] "Starting client certificate rotation" Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.090255 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.091730 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 09:17:00.096169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.092724 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 09:17:00.097044 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.097024 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 09:17:00.110582 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.110560 2562 log.go:25] "Validated CRI v1 runtime API" Jul 23 09:17:00.112633 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.112618 2562 log.go:25] "Validated CRI v1 image API" Jul 23 09:17:00.113385 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.113371 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 09:17:00.116112 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.116090 2562 fs.go:135] Filesystem UUIDs: map[5dacfe65-34bb-4869-8ea5-bfd0c8264334:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a2c8cab1-5144-46f7-9ed2-b77157aa8867:/dev/nvme0n1p4] Jul 23 09:17:00.116194 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.116113 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 09:17:00.121131 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.121014 2562 manager.go:216] Machine: {Timestamp:2026-07-23 09:17:00.119682931 +0000 UTC m=+0.329866072 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095126 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24b95847369fa88c43279b34c61190 SystemUUID:ec24b958-4736-9fa8-8c43-279b34c61190 BootID:7995ad3c-304b-4c84-b85e-577c5f031405 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:9b:18:ae:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:9b:18:ae:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:00:79:b7:f1:e7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 09:17:00.121131 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.121122 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 09:17:00.121311 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.121231 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 09:17:00.121555 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.121535 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 09:17:00.124287 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.124257 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 09:17:00.124480 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.124294 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-192.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 09:17:00.125078 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.125066 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 09:17:00.125136 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.125084 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 09:17:00.125136 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.125095 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 09:17:00.125820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.125808 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 09:17:00.127037 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.127026 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 09:17:00.130783 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130769 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 09:17:00.130914 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130903 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 09:17:00.130978 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130942 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 23 09:17:00.130978 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130958 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 09:17:00.131069 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130979 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 09:17:00.131069 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.130994 2562 kubelet.go:398] "Adding apiserver pod source" Jul 23 09:17:00.131069 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.131008 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 09:17:00.131414 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.131400 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 09:17:00.131476 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.131422 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 09:17:00.133667 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.133651 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 09:17:00.135498 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.135483 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 09:17:00.135550 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.135521 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 09:17:00.136524 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136503 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 09:17:00.136524 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136521 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136526 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136532 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136542 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136548 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136558 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136563 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136569 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136575 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136592 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 09:17:00.136597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.136597 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 09:17:00.137361 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.137351 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 09:17:00.137361 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.137361 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 09:17:00.140713 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.140682 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 09:17:00.140869 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.140852 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-192.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 09:17:00.140954 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.140914 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 09:17:00.142470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.142198 2562 server.go:1268] "Started kubelet" Jul 23 09:17:00.142470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.142285 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 09:17:00.142470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.142285 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 09:17:00.142470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.142372 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 09:17:00.142697 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.142560 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsxgc" Jul 23 09:17:00.143084 ip-10-0-132-192 systemd[1]: Started Kubernetes Kubelet. Jul 23 09:17:00.144069 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.143936 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 09:17:00.144367 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.144322 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 23 09:17:00.147904 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.146959 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-192.ec2.internal.18c4e0866352a86a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-192.ec2.internal,UID:ip-10-0-132-192.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-192.ec2.internal,},FirstTimestamp:2026-07-23 09:17:00.142159978 +0000 UTC m=+0.352343120,LastTimestamp:2026-07-23 09:17:00.142159978 +0000 UTC m=+0.352343120,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-192.ec2.internal,}" Jul 23 09:17:00.148175 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.148160 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 09:17:00.150780 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.150762 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 09:17:00.150780 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.150767 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 09:17:00.150925 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.150900 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.151526 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151503 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsxgc" Jul 23 09:17:00.151605 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151530 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 09:17:00.151686 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151668 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 09:17:00.151686 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151681 2562 factory.go:55] Registering systemd factory Jul 23 09:17:00.151977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151690 2562 factory.go:223] Registration of the systemd container factory successfully Jul 23 09:17:00.151977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151758 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 09:17:00.151977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151770 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 09:17:00.151977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151876 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 23 09:17:00.151977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.151886 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 23 09:17:00.152383 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.152367 2562 factory.go:153] Registering CRI-O factory Jul 23 09:17:00.152482 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.152398 2562 factory.go:223] Registration of the crio container factory successfully Jul 23 09:17:00.152482 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.152421 2562 factory.go:103] Registering Raw factory Jul 23 09:17:00.152482 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.152435 2562 manager.go:1196] Started watching for new ooms in manager Jul 23 09:17:00.153053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.153037 2562 manager.go:349] Starting recovery of all containers Jul 23 09:17:00.154349 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.154309 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 09:17:00.154443 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.154421 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-192.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 09:17:00.160465 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.160427 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 09:17:00.164574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.164424 2562 manager.go:354] Recovery completed Jul 23 09:17:00.169322 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.169309 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.171020 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171004 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.171095 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171033 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.171095 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171043 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.171605 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171592 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 09:17:00.171605 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171602 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 09:17:00.171727 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171607 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.171727 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.171617 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 23 09:17:00.172272 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.172256 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.172312 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.172283 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.172312 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.172293 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.173588 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.173577 2562 policy_none.go:49] "None policy: Start" Jul 23 09:17:00.173624 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.173592 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 09:17:00.173624 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.173602 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 09:17:00.175429 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.175418 2562 policy_none.go:47] "Start" Jul 23 09:17:00.216922 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.216905 2562 manager.go:337] "Starting Device Plugin manager" Jul 23 09:17:00.217015 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.216943 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 09:17:00.217015 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.216956 2562 server.go:87] "Starting device plugin registration server" Jul 23 09:17:00.217229 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.217217 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 09:17:00.217290 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.217231 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 09:17:00.217359 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.217317 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 09:17:00.217437 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.217424 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 09:17:00.217437 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.217437 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 09:17:00.217975 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.217956 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 09:17:00.218053 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.218011 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.317849 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.317782 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.320197 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.318979 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.320197 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.319018 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.320197 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.319030 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.320197 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.319064 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.325904 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.325888 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.325949 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.325913 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-192.ec2.internal\": node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.338714 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.338686 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 09:17:00.338841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.338726 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 09:17:00.338841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.338747 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 09:17:00.338841 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.338782 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 09:17:00.340581 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.340562 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:00.340758 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.340741 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.439238 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.439203 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal"] Jul 23 09:17:00.439410 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.439282 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.440274 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.440250 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.440351 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.440289 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.440351 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.440298 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.441343 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.441314 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.442411 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.442399 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.442518 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.442505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.442557 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.442538 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.443171 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443157 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.443239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443180 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.443239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443194 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.443239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443203 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.443239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443223 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.443239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.443234 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.445089 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.445076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.445136 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.445098 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 09:17:00.445774 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.445760 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientMemory" Jul 23 09:17:00.445850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.445784 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 09:17:00.445850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.445793 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeHasSufficientPID" Jul 23 09:17:00.453718 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.453703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74882c1bf505da02ef19fd26790a16c7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.ec2.internal\" (UID: \"74882c1bf505da02ef19fd26790a16c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.453776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.453727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.453776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.453746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.474866 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.474842 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-192.ec2.internal\" not found" node="ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.479007 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.478991 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-192.ec2.internal\" not found" node="ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.541733 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.541706 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.553880 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.553941 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74882c1bf505da02ef19fd26790a16c7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.ec2.internal\" (UID: \"74882c1bf505da02ef19fd26790a16c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.553941 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.553941 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553928 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.554029 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d04c7a14ae41e82132746839e64be192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal\" (UID: \"d04c7a14ae41e82132746839e64be192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.554029 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.553967 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74882c1bf505da02ef19fd26790a16c7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.ec2.internal\" (UID: \"74882c1bf505da02ef19fd26790a16c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.642496 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.642442 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.743235 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.743202 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.778241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.778218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.781435 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:00.781410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:00.843847 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.843806 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:00.950480 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:00.950445 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-192.ec2.internal\" not found" Jul 23 09:17:01.019148 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.019121 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:01.052140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.052105 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" Jul 23 09:17:01.065094 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.065060 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 09:17:01.066982 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.066964 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" Jul 23 09:17:01.077587 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.077568 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 09:17:01.080648 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.080631 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:01.085886 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.085866 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:01.089972 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.089954 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 09:17:01.090087 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.090068 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 09:17:01.090135 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.090105 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 09:17:01.090135 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.090118 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 09:17:01.090199 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.090110 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 09:17:01.131698 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.131676 2562 apiserver.go:52] "Watching apiserver" Jul 23 09:17:01.147025 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.146998 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 09:17:01.147310 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.147290 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-f75l4","openshift-image-registry/node-ca-mghn8","openshift-multus/multus-7pgwl","openshift-multus/network-metrics-daemon-nmd8t","openshift-network-diagnostics/network-check-target-zxz2t","openshift-network-operator/iptables-alerter-xk2hx","kube-system/global-pull-secret-syncer-jxtrx","kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t","openshift-dns/node-resolver-d7fjf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal","openshift-multus/multus-additional-cni-plugins-9fnfr","openshift-ovn-kubernetes/ovnkube-node-4rwpq","kube-system/konnectivity-agent-rmwhw"] Jul 23 09:17:01.149829 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.149814 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.149916 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.149898 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:01.150875 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.150858 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 09:17:01.152282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.152267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.154418 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.154398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.155103 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155079 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.155103 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155098 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 09:17:01.155251 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155114 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k8hnw\"" Jul 23 09:17:01.155251 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155101 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.155251 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155175 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 09:17:01.155251 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.155185 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 09:17:01.156466 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156422 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 09:12:01 +0000 UTC" deadline="2028-03-30 15:04:32.22072597 +0000 UTC" Jul 23 09:17:01.156466 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156465 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14789h47m31.064263671s" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-dbus\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-systemd-units\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-var-lib-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-node-log\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-config\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2715e14e-0c79-4e62-99f6-305a33da96cb-ovn-node-metrics-cert\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.156571 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.156603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156595 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-kubelet\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-systemd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-etc-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-bin\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-env-overrides\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-netns\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156855 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-ovn\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-script-lib\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-779cz\" (UniqueName: \"kubernetes.io/projected/2715e14e-0c79-4e62-99f6-305a33da96cb-kube-api-access-779cz\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-kubelet-config\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-slash\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-log-socket\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.156984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-netd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.157140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157153 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 09:17:01.157841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157310 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-974zc\"" Jul 23 09:17:01.157841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157396 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.157841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157399 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 09:17:01.157841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.157481 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.158679 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.158662 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:01.158764 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.158710 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:01.160898 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.160879 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.163166 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.163147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.163498 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.163478 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 09:17:01.163600 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.163509 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.163654 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.163609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xcctd\"" Jul 23 09:17:01.163654 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.163643 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.165618 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.165602 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.167629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.167179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 09:17:01.167629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.167327 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.167629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.167330 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pdb2j\"" Jul 23 09:17:01.167902 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.167885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.169202 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.169186 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.169489 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.169476 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.169554 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.169493 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.170063 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.170049 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p46nh\"" Jul 23 09:17:01.170256 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.170237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.170377 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.170260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 09:17:01.170484 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.170467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.170672 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.170654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vx2ms\"" Jul 23 09:17:01.171383 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.171366 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.172880 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.172855 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.173885 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.173864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r92zm\"" Jul 23 09:17:01.173991 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.173916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 09:17:01.173991 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.173946 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 09:17:01.175173 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.175154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 09:17:01.175667 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.175648 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjsk6\"" Jul 23 09:17:01.175767 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.175710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.176258 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.176235 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 09:17:01.178842 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.178825 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8pqxc\"" Jul 23 09:17:01.178950 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.178894 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 09:17:01.179175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.179161 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 09:17:01.179968 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.179953 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 09:17:01.208002 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.207983 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlvsp" Jul 23 09:17:01.218037 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.218013 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlvsp" Jul 23 09:17:01.252357 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.252334 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 09:17:01.257947 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.257926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-dbus\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.258056 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.257959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-node-log\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258056 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.257990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c85df748-f562-4ed4-a0c9-df107029989c-host\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.258056 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-agent-certs\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-node-log\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258059 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-hostroot\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-dbus\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-os-release\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-systemd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-etc-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-bin\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-run\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.258218 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-multus-daemon-config\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxc6\" (UniqueName: \"kubernetes.io/projected/8301271b-a788-4a0b-ac03-386008009cdf-kube-api-access-rmxc6\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-ovn\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258267 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-etc-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.258280 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:01.758260059 +0000 UTC m=+1.968443190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-systemd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258296 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-ovn\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-modprobe-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258332 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-bin\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm9zz\" (UniqueName: \"kubernetes.io/projected/c85df748-f562-4ed4-a0c9-df107029989c-kube-api-access-zm9zz\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-socket-dir-parent\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-kubernetes\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.258527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-bin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-conf-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-tmp\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-device-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x46dl\" (UniqueName: \"kubernetes.io/projected/46e663c0-2d50-4c18-987a-50787a03dd16-kube-api-access-x46dl\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-netns\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-script-lib\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-779cz\" (UniqueName: \"kubernetes.io/projected/2715e14e-0c79-4e62-99f6-305a33da96cb-kube-api-access-779cz\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258839 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-netns\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysconfig\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258890 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-run-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea0c7a2-8481-4f90-a089-31f955dac3b8-hosts-file\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.259320 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.258968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-sys\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-lib-modules\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-multus-certs\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx8jh\" (UniqueName: \"kubernetes.io/projected/997c0d70-352b-49c6-88c8-badfd55f82f8-kube-api-access-vx8jh\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-systemd-units\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-var-lib-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-socket-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259248 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-systemd-units\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-var-lib-openvswitch\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-env-overrides\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-registration-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-konnectivity-ca\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.260046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-system-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-cni-binary-copy\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-slash\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-netd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-etc-tuned\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-kubelet\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259572 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-slash\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259613 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-cni-netd\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259614 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcrvl\" (UniqueName: \"kubernetes.io/projected/33c5ad94-37c7-4127-91f7-9a88f64b5676-kube-api-access-lcrvl\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259620 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-script-lib\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-system-cni-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-run-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-cnibin\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-config\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259771 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2715e14e-0c79-4e62-99f6-305a33da96cb-ovn-node-metrics-cert\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259789 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-env-overrides\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.260779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-systemd\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-etc-kubernetes\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxkkl\" (UniqueName: \"kubernetes.io/projected/92ee8dcd-7f5f-4fbd-82f6-14536e167955-kube-api-access-qxkkl\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-sys-fs\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-kubelet\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-conf\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.259978 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-kubelet\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260006 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-host\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-cnibin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260063 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-os-release\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/997c0d70-352b-49c6-88c8-badfd55f82f8-iptables-alerter-script\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997c0d70-352b-49c6-88c8-badfd55f82f8-host-slash\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f96n6\" (UniqueName: \"kubernetes.io/projected/18f75276-860d-407b-9131-aeff0baabed1-kube-api-access-f96n6\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea0c7a2-8481-4f90-a089-31f955dac3b8-tmp-dir\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2715e14e-0c79-4e62-99f6-305a33da96cb-ovnkube-config\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260263 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-k8s-cni-cncf-io\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-netns\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-kubelet-config\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-log-socket\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-var-lib-kubelet\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c85df748-f562-4ed4-a0c9-df107029989c-serviceca\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1278b23a-b894-4fa2-8eef-497ee694573b-kubelet-config\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2715e14e-0c79-4e62-99f6-305a33da96cb-log-socket\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260507 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtrnq\" (UniqueName: \"kubernetes.io/projected/1ea0c7a2-8481-4f90-a089-31f955dac3b8-kube-api-access-vtrnq\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.261850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.260547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-multus\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.263471 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.263455 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2715e14e-0c79-4e62-99f6-305a33da96cb-ovn-node-metrics-cert\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.295244 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.295213 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-779cz\" (UniqueName: \"kubernetes.io/projected/2715e14e-0c79-4e62-99f6-305a33da96cb-kube-api-access-779cz\") pod \"ovnkube-node-4rwpq\" (UID: \"2715e14e-0c79-4e62-99f6-305a33da96cb\") " pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.360824 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysconfig\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.360824 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea0c7a2-8481-4f90-a089-31f955dac3b8-hosts-file\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360930 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysconfig\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-sys\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.360963 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360983 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-sys\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-lib-modules\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.361010 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:01.860994427 +0000 UTC m=+2.071177556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.360949 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1ea0c7a2-8481-4f90-a089-31f955dac3b8-hosts-file\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.361083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-multus-certs\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361090 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-lib-modules\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx8jh\" (UniqueName: \"kubernetes.io/projected/997c0d70-352b-49c6-88c8-badfd55f82f8-kube-api-access-vx8jh\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361134 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-multus-certs\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-socket-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-registration-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-konnectivity-ca\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361365 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-socket-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-system-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-cni-binary-copy\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-etc-tuned\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361453 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361481 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-etc-selinux\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361537 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-registration-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.361726 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361557 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-system-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-kubelet\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcrvl\" (UniqueName: \"kubernetes.io/projected/33c5ad94-37c7-4127-91f7-9a88f64b5676-kube-api-access-lcrvl\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-system-cni-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-cnibin\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-systemd\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-etc-kubernetes\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxkkl\" (UniqueName: \"kubernetes.io/projected/92ee8dcd-7f5f-4fbd-82f6-14536e167955-kube-api-access-qxkkl\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-sys-fs\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-conf\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-binary-copy\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-host\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361983 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-cnibin\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.361988 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-cnibin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-os-release\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362036 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-cnibin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/997c0d70-352b-49c6-88c8-badfd55f82f8-iptables-alerter-script\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.362619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997c0d70-352b-49c6-88c8-badfd55f82f8-host-slash\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-sys-fs\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-konnectivity-ca\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f96n6\" (UniqueName: \"kubernetes.io/projected/18f75276-860d-407b-9131-aeff0baabed1-kube-api-access-f96n6\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-system-cni-dir\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea0c7a2-8481-4f90-a089-31f955dac3b8-tmp-dir\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-k8s-cni-cncf-io\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362219 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-cni-binary-copy\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-sysctl-conf\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-k8s-cni-cncf-io\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-netns\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362314 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-host\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362358 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-kubelet\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-var-lib-kubelet\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-run-netns\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.363307 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ea0c7a2-8481-4f90-a089-31f955dac3b8-tmp-dir\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-etc-kubernetes\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-systemd\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362583 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997c0d70-352b-49c6-88c8-badfd55f82f8-host-slash\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362662 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-var-lib-kubelet\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-os-release\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c85df748-f562-4ed4-a0c9-df107029989c-serviceca\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtrnq\" (UniqueName: \"kubernetes.io/projected/1ea0c7a2-8481-4f90-a089-31f955dac3b8-kube-api-access-vtrnq\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.362839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-multus\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-cni-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c85df748-f562-4ed4-a0c9-df107029989c-host\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363126 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c85df748-f562-4ed4-a0c9-df107029989c-host\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-agent-certs\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/997c0d70-352b-49c6-88c8-badfd55f82f8-iptables-alerter-script\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-hostroot\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363191 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-os-release\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-multus\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364088 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-run\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-multus-daemon-config\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363284 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-run\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363308 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxc6\" (UniqueName: \"kubernetes.io/projected/8301271b-a788-4a0b-ac03-386008009cdf-kube-api-access-rmxc6\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363331 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-hostroot\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c85df748-f562-4ed4-a0c9-df107029989c-serviceca\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46e663c0-2d50-4c18-987a-50787a03dd16-os-release\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-modprobe-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363456 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-modprobe-d\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm9zz\" (UniqueName: \"kubernetes.io/projected/c85df748-f562-4ed4-a0c9-df107029989c-kube-api-access-zm9zz\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-socket-dir-parent\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-kubernetes\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-bin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-conf-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363625 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-socket-dir-parent\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-tmp\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-device-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363678 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-host-var-lib-cni-bin\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.364712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8301271b-a788-4a0b-ac03-386008009cdf-multus-conf-dir\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x46dl\" (UniqueName: \"kubernetes.io/projected/46e663c0-2d50-4c18-987a-50787a03dd16-kube-api-access-x46dl\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18f75276-860d-407b-9131-aeff0baabed1-etc-kubernetes\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.363844 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c5ad94-37c7-4127-91f7-9a88f64b5676-device-dir\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.364160 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8301271b-a788-4a0b-ac03-386008009cdf-multus-daemon-config\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.364315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.365236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.364815 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46e663c0-2d50-4c18-987a-50787a03dd16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.365841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.365818 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae-agent-certs\") pod \"konnectivity-agent-rmwhw\" (UID: \"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae\") " pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.366495 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.366475 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-tmp\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.366581 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.366516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18f75276-860d-407b-9131-aeff0baabed1-etc-tuned\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.380841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.380814 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcrvl\" (UniqueName: \"kubernetes.io/projected/33c5ad94-37c7-4127-91f7-9a88f64b5676-kube-api-access-lcrvl\") pod \"aws-ebs-csi-driver-node-c5w4t\" (UID: \"33c5ad94-37c7-4127-91f7-9a88f64b5676\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.396094 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.396066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx8jh\" (UniqueName: \"kubernetes.io/projected/997c0d70-352b-49c6-88c8-badfd55f82f8-kube-api-access-vx8jh\") pod \"iptables-alerter-xk2hx\" (UID: \"997c0d70-352b-49c6-88c8-badfd55f82f8\") " pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.406175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.406147 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxkkl\" (UniqueName: \"kubernetes.io/projected/92ee8dcd-7f5f-4fbd-82f6-14536e167955-kube-api-access-qxkkl\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.413646 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.413622 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtrnq\" (UniqueName: \"kubernetes.io/projected/1ea0c7a2-8481-4f90-a089-31f955dac3b8-kube-api-access-vtrnq\") pod \"node-resolver-d7fjf\" (UID: \"1ea0c7a2-8481-4f90-a089-31f955dac3b8\") " pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.414007 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.413985 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm9zz\" (UniqueName: \"kubernetes.io/projected/c85df748-f562-4ed4-a0c9-df107029989c-kube-api-access-zm9zz\") pod \"node-ca-mghn8\" (UID: \"c85df748-f562-4ed4-a0c9-df107029989c\") " pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.414456 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.414434 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:01.414529 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.414462 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:01.414529 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.414479 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:01.414600 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.414570 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:01.914546527 +0000 UTC m=+2.124729686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:01.415597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.415573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxc6\" (UniqueName: \"kubernetes.io/projected/8301271b-a788-4a0b-ac03-386008009cdf-kube-api-access-rmxc6\") pod \"multus-7pgwl\" (UID: \"8301271b-a788-4a0b-ac03-386008009cdf\") " pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.416065 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.416045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x46dl\" (UniqueName: \"kubernetes.io/projected/46e663c0-2d50-4c18-987a-50787a03dd16-kube-api-access-x46dl\") pod \"multus-additional-cni-plugins-9fnfr\" (UID: \"46e663c0-2d50-4c18-987a-50787a03dd16\") " pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.420007 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.419988 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f96n6\" (UniqueName: \"kubernetes.io/projected/18f75276-860d-407b-9131-aeff0baabed1-kube-api-access-f96n6\") pod \"tuned-f75l4\" (UID: \"18f75276-860d-407b-9131-aeff0baabed1\") " pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.477656 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.477621 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74882c1bf505da02ef19fd26790a16c7.slice/crio-f59cc2ae7c3a73953ff280e74c04abb169b05d1063e6593512f1cba9d364d518 WatchSource:0}: Error finding container f59cc2ae7c3a73953ff280e74c04abb169b05d1063e6593512f1cba9d364d518: Status 404 returned error can't find the container with id f59cc2ae7c3a73953ff280e74c04abb169b05d1063e6593512f1cba9d364d518 Jul 23 09:17:01.482715 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.482695 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:01.483880 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.483864 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 09:17:01.490187 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.490163 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7pgwl" Jul 23 09:17:01.492830 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.492805 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2715e14e_0c79_4e62_99f6_305a33da96cb.slice/crio-e39fba582aa2feb4b1412012358ab68e36b88bca4641902241e14102ec756dce WatchSource:0}: Error finding container e39fba582aa2feb4b1412012358ab68e36b88bca4641902241e14102ec756dce: Status 404 returned error can't find the container with id e39fba582aa2feb4b1412012358ab68e36b88bca4641902241e14102ec756dce Jul 23 09:17:01.499670 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.499644 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8301271b_a788_4a0b_ac03_386008009cdf.slice/crio-a9e0c0a246a4b8613dc86aec92fa241383ec6941184cfdc8189d84adc949cba4 WatchSource:0}: Error finding container a9e0c0a246a4b8613dc86aec92fa241383ec6941184cfdc8189d84adc949cba4: Status 404 returned error can't find the container with id a9e0c0a246a4b8613dc86aec92fa241383ec6941184cfdc8189d84adc949cba4 Jul 23 09:17:01.509680 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.509664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xk2hx" Jul 23 09:17:01.519446 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.519424 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997c0d70_352b_49c6_88c8_badfd55f82f8.slice/crio-ff6c910fcfdd941768a27abfe2d0f59cde48637929b5a5fde5239559837e07cd WatchSource:0}: Error finding container ff6c910fcfdd941768a27abfe2d0f59cde48637929b5a5fde5239559837e07cd: Status 404 returned error can't find the container with id ff6c910fcfdd941768a27abfe2d0f59cde48637929b5a5fde5239559837e07cd Jul 23 09:17:01.522536 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.522517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" Jul 23 09:17:01.529853 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.529836 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f75l4" Jul 23 09:17:01.531188 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.531169 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c5ad94_37c7_4127_91f7_9a88f64b5676.slice/crio-e6d438765248b6bbca6247bbb1cc89191cd8104dc2d4025dde0bcd85e26fea53 WatchSource:0}: Error finding container e6d438765248b6bbca6247bbb1cc89191cd8104dc2d4025dde0bcd85e26fea53: Status 404 returned error can't find the container with id e6d438765248b6bbca6247bbb1cc89191cd8104dc2d4025dde0bcd85e26fea53 Jul 23 09:17:01.532595 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.532580 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mghn8" Jul 23 09:17:01.544511 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.544489 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85df748_f562_4ed4_a0c9_df107029989c.slice/crio-4e05e46adc7e67635f0b62f3414442fd8e00292db13b5f7ed0d67afb1106382e WatchSource:0}: Error finding container 4e05e46adc7e67635f0b62f3414442fd8e00292db13b5f7ed0d67afb1106382e: Status 404 returned error can't find the container with id 4e05e46adc7e67635f0b62f3414442fd8e00292db13b5f7ed0d67afb1106382e Jul 23 09:17:01.561318 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.561296 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7fjf" Jul 23 09:17:01.570319 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.570299 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" Jul 23 09:17:01.570430 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.570405 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea0c7a2_8481_4f90_a089_31f955dac3b8.slice/crio-fc3c921c4f45bd9572aa15b472d884a58a4cd6b9365413e0f5096512cd14811e WatchSource:0}: Error finding container fc3c921c4f45bd9572aa15b472d884a58a4cd6b9365413e0f5096512cd14811e: Status 404 returned error can't find the container with id fc3c921c4f45bd9572aa15b472d884a58a4cd6b9365413e0f5096512cd14811e Jul 23 09:17:01.574439 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.574410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:01.579715 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.579695 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e663c0_2d50_4c18_987a_50787a03dd16.slice/crio-7c2ea27fdfa0ccf61c8d88548adaaab9372da099b49859c537b9e700a8f3f357 WatchSource:0}: Error finding container 7c2ea27fdfa0ccf61c8d88548adaaab9372da099b49859c537b9e700a8f3f357: Status 404 returned error can't find the container with id 7c2ea27fdfa0ccf61c8d88548adaaab9372da099b49859c537b9e700a8f3f357 Jul 23 09:17:01.584100 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:01.584079 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1ba3cc_58aa_472c_8de6_b4ffb983f8ae.slice/crio-14f418f5dd6b008857f3cce05cf7105f5d9f24844c7ee1ecd1d4d84f0e9cabcb WatchSource:0}: Error finding container 14f418f5dd6b008857f3cce05cf7105f5d9f24844c7ee1ecd1d4d84f0e9cabcb: Status 404 returned error can't find the container with id 14f418f5dd6b008857f3cce05cf7105f5d9f24844c7ee1ecd1d4d84f0e9cabcb Jul 23 09:17:01.766826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.766743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:01.766980 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.766851 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:01.766980 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.766906 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:02.766887251 +0000 UTC m=+2.977070381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:01.867591 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.867554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:01.867760 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.867693 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:01.867832 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.867766 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:02.867740202 +0000 UTC m=+3.077923339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:01.955177 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.955142 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:01.967934 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:01.967902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:01.968169 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.968147 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:01.968255 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.968177 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:01.968255 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.968192 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:01.968255 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:01.968250 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:02.968231018 +0000 UTC m=+3.178414164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:02.106004 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:02.105969 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd04c7a14ae41e82132746839e64be192.slice/crio-187bb7f05d96a65b88616fcf68104b164e41e0d87f52f6aef4770311be58d5d4 WatchSource:0}: Error finding container 187bb7f05d96a65b88616fcf68104b164e41e0d87f52f6aef4770311be58d5d4: Status 404 returned error can't find the container with id 187bb7f05d96a65b88616fcf68104b164e41e0d87f52f6aef4770311be58d5d4 Jul 23 09:17:02.222442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.222393 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 09:12:02 +0000 UTC" deadline="2028-04-21 19:53:21.741448046 +0000 UTC" Jul 23 09:17:02.222442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.222440 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15322h36m19.519013127s" Jul 23 09:17:02.342841 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.342762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:02.343007 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.342904 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:02.343339 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.343314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:02.343463 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.343440 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:02.369016 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.368952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rmwhw" event={"ID":"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae","Type":"ContainerStarted","Data":"14f418f5dd6b008857f3cce05cf7105f5d9f24844c7ee1ecd1d4d84f0e9cabcb"} Jul 23 09:17:02.380620 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.378135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerStarted","Data":"7c2ea27fdfa0ccf61c8d88548adaaab9372da099b49859c537b9e700a8f3f357"} Jul 23 09:17:02.391306 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.391267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7fjf" event={"ID":"1ea0c7a2-8481-4f90-a089-31f955dac3b8","Type":"ContainerStarted","Data":"fc3c921c4f45bd9572aa15b472d884a58a4cd6b9365413e0f5096512cd14811e"} Jul 23 09:17:02.406549 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.406508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mghn8" event={"ID":"c85df748-f562-4ed4-a0c9-df107029989c","Type":"ContainerStarted","Data":"4e05e46adc7e67635f0b62f3414442fd8e00292db13b5f7ed0d67afb1106382e"} Jul 23 09:17:02.417253 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.417214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f75l4" event={"ID":"18f75276-860d-407b-9131-aeff0baabed1","Type":"ContainerStarted","Data":"941779971e72710589fdd8a5383a3233890f5473d5e8ed52a30bde717a10d8a3"} Jul 23 09:17:02.431250 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.431148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" event={"ID":"33c5ad94-37c7-4127-91f7-9a88f64b5676","Type":"ContainerStarted","Data":"e6d438765248b6bbca6247bbb1cc89191cd8104dc2d4025dde0bcd85e26fea53"} Jul 23 09:17:02.442515 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.442480 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xk2hx" event={"ID":"997c0d70-352b-49c6-88c8-badfd55f82f8","Type":"ContainerStarted","Data":"ff6c910fcfdd941768a27abfe2d0f59cde48637929b5a5fde5239559837e07cd"} Jul 23 09:17:02.448624 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.448596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7pgwl" event={"ID":"8301271b-a788-4a0b-ac03-386008009cdf","Type":"ContainerStarted","Data":"a9e0c0a246a4b8613dc86aec92fa241383ec6941184cfdc8189d84adc949cba4"} Jul 23 09:17:02.455789 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.455760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" event={"ID":"d04c7a14ae41e82132746839e64be192","Type":"ContainerStarted","Data":"187bb7f05d96a65b88616fcf68104b164e41e0d87f52f6aef4770311be58d5d4"} Jul 23 09:17:02.469053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.469021 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"e39fba582aa2feb4b1412012358ab68e36b88bca4641902241e14102ec756dce"} Jul 23 09:17:02.477217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.477186 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" event={"ID":"74882c1bf505da02ef19fd26790a16c7","Type":"ContainerStarted","Data":"f59cc2ae7c3a73953ff280e74c04abb169b05d1063e6593512f1cba9d364d518"} Jul 23 09:17:02.742270 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.742235 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:02.780674 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.780632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:02.780844 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.780786 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:02.780913 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.780853 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:04.78083416 +0000 UTC m=+4.991017293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:02.881045 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.881006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:02.881234 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.881219 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:02.881312 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.881286 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:04.881265631 +0000 UTC m=+5.091448765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:02.881746 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.881722 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:02.983180 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:02.983055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:02.983384 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.983211 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:02.983384 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.983235 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:02.983384 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.983246 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:02.983384 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:02.983294 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:04.98328026 +0000 UTC m=+5.193463390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:03.223344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:03.223262 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 09:12:02 +0000 UTC" deadline="2028-03-02 02:29:38.322335616 +0000 UTC" Jul 23 09:17:03.223344 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:03.223325 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14105h12m35.099014833s" Jul 23 09:17:03.339630 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:03.339591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:03.339868 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:03.339756 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:04.028875 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:04.028840 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 09:17:04.339677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:04.339581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:04.339677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:04.339637 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:04.340223 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.339741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:04.340223 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.340153 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:04.798035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:04.797994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:04.798244 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.798221 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:04.798321 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.798305 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:08.798284034 +0000 UTC m=+9.008467171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:04.899692 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:04.899045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:04.899692 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.899230 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:04.899692 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:04.899296 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:08.899277926 +0000 UTC m=+9.109461071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:05.000077 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:05.000033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:05.000259 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:05.000200 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:05.000259 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:05.000224 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:05.000259 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:05.000237 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:05.000454 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:05.000309 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:09.000289835 +0000 UTC m=+9.210472986 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:05.340262 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:05.339667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:05.340262 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:05.339819 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:06.340990 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:06.340473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:06.340990 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:06.340624 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:06.340990 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:06.340703 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:06.340990 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:06.340916 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:07.339772 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:07.339727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:07.339960 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:07.339873 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:08.339492 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:08.339452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:08.340001 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.339642 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:08.340074 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:08.339453 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:08.340165 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.340129 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:08.833978 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:08.833938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:08.834182 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.834079 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:08.834182 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.834143 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:16.834123899 +0000 UTC m=+17.044307032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:08.934723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:08.934687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:08.934900 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.934820 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:08.934900 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:08.934878 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:16.934861371 +0000 UTC m=+17.145044501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:09.035492 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:09.035454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:09.035718 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:09.035697 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:09.035718 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:09.035720 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:09.035894 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:09.035729 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:09.035894 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:09.035786 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:17.03577066 +0000 UTC m=+17.245953789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:09.339619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:09.339053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:09.339619 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:09.339209 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:10.340151 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:10.340113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:10.340660 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:10.340236 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:10.341404 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:10.341377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:10.341531 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:10.341506 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:11.340034 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:11.339803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:11.340227 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:11.340161 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:12.339578 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:12.339539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:12.339769 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:12.339693 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:12.339769 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:12.339755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:12.339913 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:12.339880 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:13.339537 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:13.339498 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:13.339946 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:13.339626 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:14.339358 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:14.339317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:14.339528 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:14.339325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:14.339528 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:14.339464 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:14.339942 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:14.339522 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:15.339279 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:15.339252 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:15.339448 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:15.339376 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:16.339935 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:16.339311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:16.339935 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.339457 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:16.339935 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:16.339546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:16.339935 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.339636 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:16.897711 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:16.897671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:16.897937 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.897824 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:16.897937 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.897893 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:17:32.897873229 +0000 UTC m=+33.108056357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:16.999055 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:16.999019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:16.999240 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.999181 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:16.999302 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:16.999257 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:32.99923649 +0000 UTC m=+33.209419632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:17.099713 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:17.099680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:17.099903 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:17.099829 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:17.099903 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:17.099859 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:17.099903 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:17.099871 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:17.100054 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:17.099936 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:33.099918201 +0000 UTC m=+33.310101348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:17.339241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:17.339202 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:17.339491 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:17.339369 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:18.339474 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:18.339440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:18.339794 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:18.339483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:18.339794 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:18.339568 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:18.339794 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:18.339670 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:19.339554 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:19.339520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:19.339954 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:19.339647 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:20.339854 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:20.339810 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:20.340321 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:20.339931 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:20.340321 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:20.339988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:20.340321 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:20.340104 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:21.339694 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:21.339662 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:21.339876 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:21.339773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:22.339761 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:22.339726 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:22.340047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:22.339726 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:22.340047 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:22.339859 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:22.340047 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:22.339931 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:23.339892 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.339865 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:23.340029 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:23.340008 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:23.529602 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.529391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" event={"ID":"74882c1bf505da02ef19fd26790a16c7","Type":"ContainerStarted","Data":"937b0b75346d3d23ced10d04e89e468538bec83e99686d31c47744d9074da2a9"} Jul 23 09:17:23.534125 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.533702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f75l4" event={"ID":"18f75276-860d-407b-9131-aeff0baabed1","Type":"ContainerStarted","Data":"7d54acc81a5abc37bb42c8bd93581a71e84e58e210fb08fa333065d941181895"} Jul 23 09:17:23.535465 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.535430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7pgwl" event={"ID":"8301271b-a788-4a0b-ac03-386008009cdf","Type":"ContainerStarted","Data":"2ae9c73934b4b6e3f1e3561c71a7576d98b4ecf0acc3d7871b0e6c640b570e75"} Jul 23 09:17:23.537553 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.537528 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:17:23.538568 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.538541 2562 generic.go:356] "Generic (PLEG): container finished" podID="2715e14e-0c79-4e62-99f6-305a33da96cb" containerID="c3cfd394f869a6a70fce90573a3aa2eed3dd11f2dcc1ab7371fca27ca575df86" exitCode=1 Jul 23 09:17:23.538662 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.538585 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"2ae1aa8d21e6512ddf29be600b3a3a38b66fb645947293b909c5b462bc5ec89c"} Jul 23 09:17:23.538662 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.538655 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerDied","Data":"c3cfd394f869a6a70fce90573a3aa2eed3dd11f2dcc1ab7371fca27ca575df86"} Jul 23 09:17:23.538765 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.538704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"45e97ded79054dab0307662eb9abc20c850c08a979512e745f4caf331c4eaf10"} Jul 23 09:17:23.552358 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.552299 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.ec2.internal" podStartSLOduration=21.552284825 podStartE2EDuration="21.552284825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:23.551900759 +0000 UTC m=+23.762083910" watchObservedRunningTime="2026-07-23 09:17:23.552284825 +0000 UTC m=+23.762467974" Jul 23 09:17:23.603581 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.603522 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f75l4" podStartSLOduration=1.456876576 podStartE2EDuration="22.603500526s" totalImagesPullingTime="21.14662395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:23.576617819 +0000 UTC m=+23.786800973" watchObservedRunningTime="2026-07-23 09:17:23.603500526 +0000 UTC m=+23.813683678" Jul 23 09:17:23.603785 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:23.603726 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7pgwl" podStartSLOduration=1.38102126 podStartE2EDuration="22.603688535s" totalImagesPullingTime="21.222667275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:23.602860201 +0000 UTC m=+23.813043353" watchObservedRunningTime="2026-07-23 09:17:23.603688535 +0000 UTC m=+23.813871685" Jul 23 09:17:24.339779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.339540 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:24.339935 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.339541 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:24.339935 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:24.339812 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:24.339935 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:24.339920 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:24.542850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.542813 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rmwhw" event={"ID":"3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae","Type":"ContainerStarted","Data":"984ce2cc6b65f2b24398f0c984562080751b99cbaf5ee58caa4997318df1aaf8"} Jul 23 09:17:24.544485 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.544460 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="45147b48ca3808c8fae1613f558be6134da13634cfec9573fd3461fe7818fe80" exitCode=0 Jul 23 09:17:24.544587 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.544530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"45147b48ca3808c8fae1613f558be6134da13634cfec9573fd3461fe7818fe80"} Jul 23 09:17:24.549046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.549017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7fjf" event={"ID":"1ea0c7a2-8481-4f90-a089-31f955dac3b8","Type":"ContainerStarted","Data":"74ab130ef3ff2f0ba78622b8dab414e9781d886dd981f35e684ed083177fca5e"} Jul 23 09:17:24.550907 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.550885 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mghn8" event={"ID":"c85df748-f562-4ed4-a0c9-df107029989c","Type":"ContainerStarted","Data":"1553dd3113c9d60e573e9a1b8449028bc37141bace655528a560cee958ea2a8c"} Jul 23 09:17:24.552374 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.552351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" event={"ID":"33c5ad94-37c7-4127-91f7-9a88f64b5676","Type":"ContainerStarted","Data":"d68bda58b5fb5dbb6613d96746f820acf506c6f604bbcabf77ccae9e770e82f3"} Jul 23 09:17:24.553870 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.553845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xk2hx" event={"ID":"997c0d70-352b-49c6-88c8-badfd55f82f8","Type":"ContainerStarted","Data":"9cee16e1fd482799a6c91f5e9898c9e02378e5181212b486ac4280de6731ad78"} Jul 23 09:17:24.555237 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.555217 2562 generic.go:356] "Generic (PLEG): container finished" podID="d04c7a14ae41e82132746839e64be192" containerID="eaee683cf4b22c7fc65f7618d4b234bad1e6edaf00a7aa76a96eaa441ae418cf" exitCode=0 Jul 23 09:17:24.555313 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.555283 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" event={"ID":"d04c7a14ae41e82132746839e64be192","Type":"ContainerDied","Data":"eaee683cf4b22c7fc65f7618d4b234bad1e6edaf00a7aa76a96eaa441ae418cf"} Jul 23 09:17:24.560123 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.560102 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:17:24.560677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.560649 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"56079548d48a82fedd8a4f8b4c9f8bec100386349e14bc2f291830a127b669d5"} Jul 23 09:17:24.560756 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.560728 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"88a5c980111f77fef14bb6887ed5a713ff0ee959eb6e83aa251d63e99c966e23"} Jul 23 09:17:24.560826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.560782 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"7890f01868431d1b0588a76adab03fb01bba45d9a567ba863f6ad1aa0824c8e3"} Jul 23 09:17:24.565373 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.565314 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rmwhw" podStartSLOduration=2.462701326 podStartE2EDuration="23.56529794s" totalImagesPullingTime="21.102596614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:24.565206792 +0000 UTC m=+24.775389944" watchObservedRunningTime="2026-07-23 09:17:24.56529794 +0000 UTC m=+24.775481095" Jul 23 09:17:24.644729 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.644686 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xk2hx" podStartSLOduration=2.4770528609999998 podStartE2EDuration="23.644670567s" totalImagesPullingTime="21.167617706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:24.644484903 +0000 UTC m=+24.854668053" watchObservedRunningTime="2026-07-23 09:17:24.644670567 +0000 UTC m=+24.854853717" Jul 23 09:17:24.669558 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:24.669514 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mghn8" podStartSLOduration=2.527594496 podStartE2EDuration="23.669497924s" totalImagesPullingTime="21.141903428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:24.669305302 +0000 UTC m=+24.879488452" watchObservedRunningTime="2026-07-23 09:17:24.669497924 +0000 UTC m=+24.879681074" Jul 23 09:17:25.339663 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:25.339626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:25.339975 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:25.339781 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:25.564257 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:25.564228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" event={"ID":"d04c7a14ae41e82132746839e64be192","Type":"ContainerStarted","Data":"60488221fc03c091219cd16592e2901519d89df77bb0aee71dba1951ce851a74"} Jul 23 09:17:25.600029 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:25.599908 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d7fjf" podStartSLOduration=3.4826359829999998 podStartE2EDuration="24.599884937s" totalImagesPullingTime="21.117248954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:24.696507185 +0000 UTC m=+24.906690336" watchObservedRunningTime="2026-07-23 09:17:25.599884937 +0000 UTC m=+25.810068082" Jul 23 09:17:25.600192 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:25.600065 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.ec2.internal" podStartSLOduration=24.600052675 podStartE2EDuration="24.600052675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:25.599580007 +0000 UTC m=+25.809763158" watchObservedRunningTime="2026-07-23 09:17:25.600052675 +0000 UTC m=+25.810235826" Jul 23 09:17:25.669027 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:25.668999 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 09:17:26.229725 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.229617 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T09:17:25.669018783Z","UUID":"3702cccf-78b2-44d4-a82e-9de346530870","Handler":null,"Name":"","Endpoint":""} Jul 23 09:17:26.231609 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.231581 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 09:17:26.231754 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.231625 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 09:17:26.340125 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.340091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:26.340125 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.340131 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:26.340399 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:26.340230 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:26.340460 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:26.340394 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:26.569789 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.569698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" event={"ID":"33c5ad94-37c7-4127-91f7-9a88f64b5676","Type":"ContainerStarted","Data":"c1a6557ef4a48028cf6929847f9b3d6dcfc3cce946bf8b9280bbb6ae51af3988"} Jul 23 09:17:26.573776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.573750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:17:26.574233 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:26.574208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"206e6adfc2793bf26cebe4318f697655df9c9e2af2c3a50ae1b784607c4efb76"} Jul 23 09:17:27.339833 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:27.339801 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:27.340019 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:27.339913 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:27.578611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:27.578560 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" event={"ID":"33c5ad94-37c7-4127-91f7-9a88f64b5676","Type":"ContainerStarted","Data":"713011981e143702fd42a22c90de60d097bc90604827993fe0e94c64e677f792"} Jul 23 09:17:27.598085 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:27.597975 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c5w4t" podStartSLOduration=2.745104794 podStartE2EDuration="26.597950768s" totalImagesPullingTime="23.852845974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:27.597919632 +0000 UTC m=+27.808102794" watchObservedRunningTime="2026-07-23 09:17:27.597950768 +0000 UTC m=+27.808133920" Jul 23 09:17:28.340244 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:28.339954 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:28.340443 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:28.339997 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:28.340443 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:28.340322 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:28.340443 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:28.340396 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:28.728254 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:28.728226 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:28.728789 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:28.728770 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:29.339661 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.339579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:29.339814 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:29.339701 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:29.586162 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:17:29.586547 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"ded78927fb40b74e2255561ae810a273c0f92240b56d191b9ab83af8c95979cc"} Jul 23 09:17:29.586803 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:29.586926 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586844 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:29.586926 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586883 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:29.587042 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.586928 2562 scope.go:117] "RemoveContainer" containerID="c3cfd394f869a6a70fce90573a3aa2eed3dd11f2dcc1ab7371fca27ca575df86" Jul 23 09:17:29.588720 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.588696 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="b519bcbb8dbf93dc6cb6fafa1c4a5458ffbcab9f559d1f792c652028c7720a92" exitCode=0 Jul 23 09:17:29.588830 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.588786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"b519bcbb8dbf93dc6cb6fafa1c4a5458ffbcab9f559d1f792c652028c7720a92"} Jul 23 09:17:29.589044 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.589026 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:29.589926 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.589878 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rmwhw" Jul 23 09:17:29.603765 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.603745 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:29.604236 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:29.604218 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:17:30.340846 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:30.340671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:30.341795 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:30.340745 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:30.341795 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:30.340904 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:30.342147 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:30.342113 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:30.595399 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:30.595290 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:17:30.595742 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:30.595714 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" event={"ID":"2715e14e-0c79-4e62-99f6-305a33da96cb","Type":"ContainerStarted","Data":"9cad77dd823fb69310a6767566f3a9f4e70eda6ba69a4ea1ae1094ab9851434c"} Jul 23 09:17:30.628637 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:30.628586 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" podStartSLOduration=8.016779983 podStartE2EDuration="29.628571416s" totalImagesPullingTime="21.611791433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:30.628204108 +0000 UTC m=+30.838387259" watchObservedRunningTime="2026-07-23 09:17:30.628571416 +0000 UTC m=+30.838754566" Jul 23 09:17:31.102304 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.102262 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jxtrx"] Jul 23 09:17:31.102492 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.102388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:31.102492 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:31.102476 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:31.105255 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.105231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zxz2t"] Jul 23 09:17:31.105403 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.105321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:31.105473 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:31.105431 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:31.105802 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.105786 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nmd8t"] Jul 23 09:17:31.105890 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.105877 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:31.105973 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:31.105957 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:31.600021 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.599982 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="db71fdd66a8aaa863013fee892a69ce07b240f6bd4529b7ec114f44550f6f7b0" exitCode=0 Jul 23 09:17:31.600441 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:31.600066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"db71fdd66a8aaa863013fee892a69ce07b240f6bd4529b7ec114f44550f6f7b0"} Jul 23 09:17:32.342954 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:32.342929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:32.343116 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:32.343023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:32.916225 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:32.916177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:32.916665 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:32.916293 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:32.916665 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:32.916372 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret podName:1278b23a-b894-4fa2-8eef-497ee694573b nodeName:}" failed. No retries permitted until 2026-07-23 09:18:04.916359314 +0000 UTC m=+65.126542442 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret") pod "global-pull-secret-syncer-jxtrx" (UID: "1278b23a-b894-4fa2-8eef-497ee694573b") : object "kube-system"/"original-pull-secret" not registered Jul 23 09:17:33.016516 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.016477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:33.016665 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.016590 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:33.016730 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.016675 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs podName:92ee8dcd-7f5f-4fbd-82f6-14536e167955 nodeName:}" failed. No retries permitted until 2026-07-23 09:18:05.016660907 +0000 UTC m=+65.226844036 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs") pod "network-metrics-daemon-nmd8t" (UID: "92ee8dcd-7f5f-4fbd-82f6-14536e167955") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 09:17:33.117110 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.117026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:33.117268 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.117206 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 09:17:33.117268 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.117230 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 09:17:33.117268 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.117241 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zcsmt for pod openshift-network-diagnostics/network-check-target-zxz2t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:33.117409 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.117293 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt podName:ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09 nodeName:}" failed. No retries permitted until 2026-07-23 09:18:05.117278574 +0000 UTC m=+65.327461702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcsmt" (UniqueName: "kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt") pod "network-check-target-zxz2t" (UID: "ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 09:17:33.339931 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.339901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:33.340105 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.340001 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:33.340105 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.340050 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:33.340181 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:33.340138 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:33.610478 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.610432 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="1a110b13db11eb7c406b29f9ff1b60b924eff96726e35afe0a5dfd996eff648a" exitCode=0 Jul 23 09:17:33.610641 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:33.610512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"1a110b13db11eb7c406b29f9ff1b60b924eff96726e35afe0a5dfd996eff648a"} Jul 23 09:17:34.343101 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:34.343070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:34.343525 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:34.343209 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jxtrx" podUID="1278b23a-b894-4fa2-8eef-497ee694573b" Jul 23 09:17:35.339551 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:35.339498 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:35.339739 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:35.339525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:35.339739 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:35.339642 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zxz2t" podUID="ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09" Jul 23 09:17:35.339831 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:35.339800 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nmd8t" podUID="92ee8dcd-7f5f-4fbd-82f6-14536e167955" Jul 23 09:17:36.076479 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.076235 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-192.ec2.internal" event="NodeReady" Jul 23 09:17:36.076877 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.076554 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 09:17:36.140232 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.140193 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cvxp6"] Jul 23 09:17:36.142845 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.142811 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lgmrh"] Jul 23 09:17:36.142976 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.142932 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.144810 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.144784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.146261 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.146241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.147704 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.147878 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8qpt\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.148392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.148630 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.149072 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 09:17:36.150058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.149632 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bcmmh\"" Jul 23 09:17:36.159488 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.159455 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cvxp6"] Jul 23 09:17:36.171571 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.171534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lgmrh"] Jul 23 09:17:36.241722 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-cert\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.241722 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56473cc-da76-4b86-b727-322ff452df75-tmp-dir\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.241964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tkpm\" (UniqueName: \"kubernetes.io/projected/c56473cc-da76-4b86-b727-322ff452df75-kube-api-access-5tkpm\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.241964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241814 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tn9\" (UniqueName: \"kubernetes.io/projected/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-kube-api-access-f2tn9\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.241964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c56473cc-da76-4b86-b727-322ff452df75-config-volume\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.241964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.241918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.339831 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.339738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:17:36.342210 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tkpm\" (UniqueName: \"kubernetes.io/projected/c56473cc-da76-4b86-b727-322ff452df75-kube-api-access-5tkpm\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.342370 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tn9\" (UniqueName: \"kubernetes.io/projected/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-kube-api-access-f2tn9\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.342370 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c56473cc-da76-4b86-b727-322ff452df75-config-volume\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.342370 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.342543 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-cert\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.342543 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56473cc-da76-4b86-b727-322ff452df75-tmp-dir\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.342806 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342772 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 09:17:36.342886 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.342803 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c56473cc-da76-4b86-b727-322ff452df75-tmp-dir\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.342942 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:36.342901 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:17:36.342992 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:36.342962 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls podName:c56473cc-da76-4b86-b727-322ff452df75 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:36.842937785 +0000 UTC m=+37.053120919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls") pod "dns-default-cvxp6" (UID: "c56473cc-da76-4b86-b727-322ff452df75") : secret "dns-default-metrics-tls" not found Jul 23 09:17:36.343288 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.343266 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c56473cc-da76-4b86-b727-322ff452df75-config-volume\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.348617 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.348594 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-cert\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.352298 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.352216 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tkpm\" (UniqueName: \"kubernetes.io/projected/c56473cc-da76-4b86-b727-322ff452df75-kube-api-access-5tkpm\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.352441 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.352320 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tn9\" (UniqueName: \"kubernetes.io/projected/300ce848-6c8c-4028-b5fd-3d4b4d5fad21-kube-api-access-f2tn9\") pod \"ingress-canary-lgmrh\" (UID: \"300ce848-6c8c-4028-b5fd-3d4b4d5fad21\") " pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.464411 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.464380 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lgmrh" Jul 23 09:17:36.846065 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:36.846023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:36.846286 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:36.846172 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:17:36.846286 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:36.846247 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls podName:c56473cc-da76-4b86-b727-322ff452df75 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:37.846226212 +0000 UTC m=+38.056409344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls") pod "dns-default-cvxp6" (UID: "c56473cc-da76-4b86-b727-322ff452df75") : secret "dns-default-metrics-tls" not found Jul 23 09:17:37.158466 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.158435 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kj2j4"] Jul 23 09:17:37.160962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.160942 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.165481 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.165462 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 09:17:37.166337 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.166309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 09:17:37.166446 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.166348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 09:17:37.166446 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.166348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sjj26\"" Jul 23 09:17:37.166446 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.166355 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 09:17:37.178265 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.178233 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kj2j4"] Jul 23 09:17:37.249878 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.249833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.250075 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.249890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csfq9\" (UniqueName: \"kubernetes.io/projected/801082ef-11db-4f35-86fc-3806e65015cf-kube-api-access-csfq9\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.250075 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.249913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/801082ef-11db-4f35-86fc-3806e65015cf-crio-socket\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.250075 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.249959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/801082ef-11db-4f35-86fc-3806e65015cf-data-volume\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.250075 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.250027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.339839 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.339806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:17:37.340033 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.339842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:17:37.342908 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.342878 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 09:17:37.343061 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.342913 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tghcj\"" Jul 23 09:17:37.343061 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.342950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s2sdc\"" Jul 23 09:17:37.343209 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.343161 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 09:17:37.343318 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.343299 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 09:17:37.350381 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350510 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.350440 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:17:37.350510 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350510 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csfq9\" (UniqueName: \"kubernetes.io/projected/801082ef-11db-4f35-86fc-3806e65015cf-kube-api-access-csfq9\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350510 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.350492 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:37.850476723 +0000 UTC m=+38.060659851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : secret "insights-runtime-extractor-tls" not found Jul 23 09:17:37.350704 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/801082ef-11db-4f35-86fc-3806e65015cf-crio-socket\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350704 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.350568 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:17:37.350704 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/801082ef-11db-4f35-86fc-3806e65015cf-data-volume\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350704 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.350621 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:37.850606405 +0000 UTC m=+38.060789533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : configmap "kube-rbac-proxy" not found Jul 23 09:17:37.350864 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/801082ef-11db-4f35-86fc-3806e65015cf-crio-socket\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.350911 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.350879 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/801082ef-11db-4f35-86fc-3806e65015cf-data-volume\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.360052 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.360029 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csfq9\" (UniqueName: \"kubernetes.io/projected/801082ef-11db-4f35-86fc-3806e65015cf-kube-api-access-csfq9\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.854514 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.854300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.854659 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854470 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:17:37.854776 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854669 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:17:37.854776 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854699 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:38.854673417 +0000 UTC m=+39.064856545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : secret "insights-runtime-extractor-tls" not found Jul 23 09:17:37.854776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.854609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:37.854776 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854724 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:38.854707941 +0000 UTC m=+39.064891086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : configmap "kube-rbac-proxy" not found Jul 23 09:17:37.854916 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.854783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:37.854916 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854878 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:17:37.854983 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:37.854920 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls podName:c56473cc-da76-4b86-b727-322ff452df75 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:39.854908239 +0000 UTC m=+40.065091372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls") pod "dns-default-cvxp6" (UID: "c56473cc-da76-4b86-b727-322ff452df75") : secret "dns-default-metrics-tls" not found Jul 23 09:17:37.855493 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.855468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lgmrh"] Jul 23 09:17:37.857376 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:37.857317 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod300ce848_6c8c_4028_b5fd_3d4b4d5fad21.slice/crio-a7da4e073ed4763a18d5a6f7b1021008a850faba07c1787f4c52b02f4d540630 WatchSource:0}: Error finding container a7da4e073ed4763a18d5a6f7b1021008a850faba07c1787f4c52b02f4d540630: Status 404 returned error can't find the container with id a7da4e073ed4763a18d5a6f7b1021008a850faba07c1787f4c52b02f4d540630 Jul 23 09:17:37.865013 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.863350 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg"] Jul 23 09:17:37.867071 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.867046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" Jul 23 09:17:37.871025 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.871002 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 09:17:37.871242 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.871224 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p7g9r\"" Jul 23 09:17:37.871387 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.871295 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 09:17:37.882398 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.882369 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg"] Jul 23 09:17:37.955755 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:37.955730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v62kl\" (UniqueName: \"kubernetes.io/projected/4a245755-f466-47aa-af41-2eb04a26f9ed-kube-api-access-v62kl\") pod \"migrator-7cc9c7ffd7-j27xg\" (UID: \"4a245755-f466-47aa-af41-2eb04a26f9ed\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" Jul 23 09:17:38.056470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.056427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v62kl\" (UniqueName: \"kubernetes.io/projected/4a245755-f466-47aa-af41-2eb04a26f9ed-kube-api-access-v62kl\") pod \"migrator-7cc9c7ffd7-j27xg\" (UID: \"4a245755-f466-47aa-af41-2eb04a26f9ed\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" Jul 23 09:17:38.069091 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.069056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v62kl\" (UniqueName: \"kubernetes.io/projected/4a245755-f466-47aa-af41-2eb04a26f9ed-kube-api-access-v62kl\") pod \"migrator-7cc9c7ffd7-j27xg\" (UID: \"4a245755-f466-47aa-af41-2eb04a26f9ed\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" Jul 23 09:17:38.176952 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.176919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" Jul 23 09:17:38.320229 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.320199 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg"] Jul 23 09:17:38.323192 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:38.323172 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a245755_f466_47aa_af41_2eb04a26f9ed.slice/crio-b948d536fce233d425770b174e1c7f9a72260883f115da77a396f892c18538b8 WatchSource:0}: Error finding container b948d536fce233d425770b174e1c7f9a72260883f115da77a396f892c18538b8: Status 404 returned error can't find the container with id b948d536fce233d425770b174e1c7f9a72260883f115da77a396f892c18538b8 Jul 23 09:17:38.624356 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.624254 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" event={"ID":"4a245755-f466-47aa-af41-2eb04a26f9ed","Type":"ContainerStarted","Data":"b948d536fce233d425770b174e1c7f9a72260883f115da77a396f892c18538b8"} Jul 23 09:17:38.625673 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.625638 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lgmrh" event={"ID":"300ce848-6c8c-4028-b5fd-3d4b4d5fad21","Type":"ContainerStarted","Data":"a7da4e073ed4763a18d5a6f7b1021008a850faba07c1787f4c52b02f4d540630"} Jul 23 09:17:38.628910 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.628881 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="2ea8b6f5c35ad7fb564bb6ef9497d0ce4a237c604d6018395fb75ab861ce1cce" exitCode=0 Jul 23 09:17:38.629085 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.628959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"2ea8b6f5c35ad7fb564bb6ef9497d0ce4a237c604d6018395fb75ab861ce1cce"} Jul 23 09:17:38.862906 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.862869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:38.863111 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:38.862955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:38.863111 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:38.863038 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:17:38.863111 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:38.863051 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:17:38.863269 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:38.863119 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:40.863096824 +0000 UTC m=+41.073279954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : configmap "kube-rbac-proxy" not found Jul 23 09:17:38.863269 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:38.863140 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:40.863131065 +0000 UTC m=+41.073314217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : secret "insights-runtime-extractor-tls" not found Jul 23 09:17:39.029710 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:39.029674 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7fjf_1ea0c7a2-8481-4f90-a089-31f955dac3b8/dns-node-resolver/0.log" Jul 23 09:17:39.636444 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:39.636404 2562 generic.go:356] "Generic (PLEG): container finished" podID="46e663c0-2d50-4c18-987a-50787a03dd16" containerID="5f2165d99e1442d1340ad70621d1b9e21e969610bf51e50b69db762232250709" exitCode=0 Jul 23 09:17:39.636894 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:39.636457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerDied","Data":"5f2165d99e1442d1340ad70621d1b9e21e969610bf51e50b69db762232250709"} Jul 23 09:17:39.872346 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:39.872299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:39.872552 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:39.872450 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:17:39.872552 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:39.872518 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls podName:c56473cc-da76-4b86-b727-322ff452df75 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:43.872501865 +0000 UTC m=+44.082684993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls") pod "dns-default-cvxp6" (UID: "c56473cc-da76-4b86-b727-322ff452df75") : secret "dns-default-metrics-tls" not found Jul 23 09:17:40.230245 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.230218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mghn8_c85df748-f562-4ed4-a0c9-df107029989c/node-ca/0.log" Jul 23 09:17:40.368700 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.368669 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9gb8w"] Jul 23 09:17:40.380028 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.379998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.381585 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.381561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9gb8w"] Jul 23 09:17:40.382720 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.382678 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 09:17:40.382848 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.382816 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 09:17:40.383019 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.382999 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 09:17:40.383663 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.383644 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kr2kb\"" Jul 23 09:17:40.383757 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.383667 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 09:17:40.476837 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.476806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxqxt\" (UniqueName: \"kubernetes.io/projected/b763dea1-a9b5-471e-abb1-e4b73f763b99-kube-api-access-cxqxt\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.477005 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.476862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-key\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.477005 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.476966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-cabundle\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.578156 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.578125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxqxt\" (UniqueName: \"kubernetes.io/projected/b763dea1-a9b5-471e-abb1-e4b73f763b99-kube-api-access-cxqxt\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.578234 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.578222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-key\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.578311 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.578295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-cabundle\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.579083 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.579060 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-cabundle\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.581979 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.581947 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b763dea1-a9b5-471e-abb1-e4b73f763b99-signing-key\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.587396 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.587364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxqxt\" (UniqueName: \"kubernetes.io/projected/b763dea1-a9b5-471e-abb1-e4b73f763b99-kube-api-access-cxqxt\") pod \"service-ca-58598695f7-9gb8w\" (UID: \"b763dea1-a9b5-471e-abb1-e4b73f763b99\") " pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.641909 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.641876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" event={"ID":"4a245755-f466-47aa-af41-2eb04a26f9ed","Type":"ContainerStarted","Data":"b27155a8e583259ee38e33c64b2627e3ce26da8c75ce26f007f9a8c5bf5239a6"} Jul 23 09:17:40.643813 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.643452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lgmrh" event={"ID":"300ce848-6c8c-4028-b5fd-3d4b4d5fad21","Type":"ContainerStarted","Data":"1c1d412a657d9b84adaf9a3b0cf2f5413fd186ca71ea1a62a2f3adeaf00fa4cb"} Jul 23 09:17:40.646508 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.646487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" event={"ID":"46e663c0-2d50-4c18-987a-50787a03dd16","Type":"ContainerStarted","Data":"b93ee2c2fb0150c975846dfe0d5081227a84203557938d31b2ef0ec300bf010e"} Jul 23 09:17:40.660610 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.660570 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lgmrh" podStartSLOduration=2.027716265 podStartE2EDuration="4.660555236s" totalImagesPullingTime="2.632838971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:40.659767571 +0000 UTC m=+40.869950722" watchObservedRunningTime="2026-07-23 09:17:40.660555236 +0000 UTC m=+40.870738383" Jul 23 09:17:40.683538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.683493 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9fnfr" podStartSLOduration=8.022809517 podStartE2EDuration="39.683474203s" totalImagesPullingTime="31.660664686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:40.682315095 +0000 UTC m=+40.892498245" watchObservedRunningTime="2026-07-23 09:17:40.683474203 +0000 UTC m=+40.893657353" Jul 23 09:17:40.690188 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.690162 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-9gb8w" Jul 23 09:17:40.846486 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.846457 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9gb8w"] Jul 23 09:17:40.849468 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:40.849444 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb763dea1_a9b5_471e_abb1_e4b73f763b99.slice/crio-1f92b50bab9430aa4719a10b6291926baf21b6cc613c291f51bfa704601b4d72 WatchSource:0}: Error finding container 1f92b50bab9430aa4719a10b6291926baf21b6cc613c291f51bfa704601b4d72: Status 404 returned error can't find the container with id 1f92b50bab9430aa4719a10b6291926baf21b6cc613c291f51bfa704601b4d72 Jul 23 09:17:40.881643 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.881609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:40.881813 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:40.881677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:40.884820 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:40.882087 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 09:17:40.884820 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:40.882136 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:17:40.884820 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:40.882175 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:44.882154579 +0000 UTC m=+45.092337720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : configmap "kube-rbac-proxy" not found Jul 23 09:17:40.884820 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:40.882215 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:44.882196756 +0000 UTC m=+45.092379903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : secret "insights-runtime-extractor-tls" not found Jul 23 09:17:41.231809 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:41.231774 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lgmrh_300ce848-6c8c-4028-b5fd-3d4b4d5fad21/serve-healthcheck-canary/0.log" Jul 23 09:17:41.651539 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:41.651439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" event={"ID":"4a245755-f466-47aa-af41-2eb04a26f9ed","Type":"ContainerStarted","Data":"16215ac73151afef06ab15f7f56f8ca23773391baf1a8c989362f6d291be6362"} Jul 23 09:17:41.652938 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:41.652909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-9gb8w" event={"ID":"b763dea1-a9b5-471e-abb1-e4b73f763b99","Type":"ContainerStarted","Data":"1f92b50bab9430aa4719a10b6291926baf21b6cc613c291f51bfa704601b4d72"} Jul 23 09:17:41.676397 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:41.676316 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-j27xg" podStartSLOduration=2.514970506 podStartE2EDuration="4.676297989s" totalImagesPullingTime="2.161327483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:41.675532758 +0000 UTC m=+41.885715909" watchObservedRunningTime="2026-07-23 09:17:41.676297989 +0000 UTC m=+41.886481140" Jul 23 09:17:43.660620 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:43.660586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-9gb8w" event={"ID":"b763dea1-a9b5-471e-abb1-e4b73f763b99","Type":"ContainerStarted","Data":"b48e856457fae1974a9f24dfd9e59f05c4dd11e8dc8ad83a858f282307a99d09"} Jul 23 09:17:43.688053 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:43.687990 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-9gb8w" podStartSLOduration=1.048002706 podStartE2EDuration="3.687967364s" totalImagesPullingTime="2.639964658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:43.68715938 +0000 UTC m=+43.897342531" watchObservedRunningTime="2026-07-23 09:17:43.687967364 +0000 UTC m=+43.898150515" Jul 23 09:17:43.907212 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:43.907178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:43.907422 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:43.907312 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 09:17:43.907422 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:43.907394 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls podName:c56473cc-da76-4b86-b727-322ff452df75 nodeName:}" failed. No retries permitted until 2026-07-23 09:17:51.907378555 +0000 UTC m=+52.117561686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls") pod "dns-default-cvxp6" (UID: "c56473cc-da76-4b86-b727-322ff452df75") : secret "dns-default-metrics-tls" not found Jul 23 09:17:44.913896 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:44.913867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:44.914283 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:44.913945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:44.914283 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:44.914026 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 09:17:44.914283 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:17:44.914095 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls podName:801082ef-11db-4f35-86fc-3806e65015cf nodeName:}" failed. No retries permitted until 2026-07-23 09:17:52.914076503 +0000 UTC m=+53.124259633 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kj2j4" (UID: "801082ef-11db-4f35-86fc-3806e65015cf") : secret "insights-runtime-extractor-tls" not found Jul 23 09:17:44.914602 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:44.914583 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/801082ef-11db-4f35-86fc-3806e65015cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:51.967849 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:51.967812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:51.970278 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:51.970257 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c56473cc-da76-4b86-b727-322ff452df75-metrics-tls\") pod \"dns-default-cvxp6\" (UID: \"c56473cc-da76-4b86-b727-322ff452df75\") " pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:52.057442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:52.057408 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:52.240067 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:52.239983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cvxp6"] Jul 23 09:17:52.244442 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:52.244416 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56473cc_da76_4b86_b727_322ff452df75.slice/crio-8dfebda1dc897fbd464b8ec7633777110849467d0fad940da304547496bb3205 WatchSource:0}: Error finding container 8dfebda1dc897fbd464b8ec7633777110849467d0fad940da304547496bb3205: Status 404 returned error can't find the container with id 8dfebda1dc897fbd464b8ec7633777110849467d0fad940da304547496bb3205 Jul 23 09:17:52.685946 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:52.685916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvxp6" event={"ID":"c56473cc-da76-4b86-b727-322ff452df75","Type":"ContainerStarted","Data":"8dfebda1dc897fbd464b8ec7633777110849467d0fad940da304547496bb3205"} Jul 23 09:17:52.975765 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:52.975671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:52.978776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:52.978744 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/801082ef-11db-4f35-86fc-3806e65015cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kj2j4\" (UID: \"801082ef-11db-4f35-86fc-3806e65015cf\") " pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:53.072492 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:53.072452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kj2j4" Jul 23 09:17:53.239878 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:53.239798 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kj2j4"] Jul 23 09:17:53.243299 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:17:53.243268 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801082ef_11db_4f35_86fc_3806e65015cf.slice/crio-a8c3f810c9bac59bc4b589c4d7e7692495f835260f808de49963d11f298c5d5d WatchSource:0}: Error finding container a8c3f810c9bac59bc4b589c4d7e7692495f835260f808de49963d11f298c5d5d: Status 404 returned error can't find the container with id a8c3f810c9bac59bc4b589c4d7e7692495f835260f808de49963d11f298c5d5d Jul 23 09:17:53.690964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:53.690926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kj2j4" event={"ID":"801082ef-11db-4f35-86fc-3806e65015cf","Type":"ContainerStarted","Data":"a8c3f810c9bac59bc4b589c4d7e7692495f835260f808de49963d11f298c5d5d"} Jul 23 09:17:55.699839 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:55.699800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvxp6" event={"ID":"c56473cc-da76-4b86-b727-322ff452df75","Type":"ContainerStarted","Data":"9d7f72aaf65eb9f1d2083e827de6e8989364d82e430445357dcba513b78fc656"} Jul 23 09:17:55.700287 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:55.699859 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cvxp6" event={"ID":"c56473cc-da76-4b86-b727-322ff452df75","Type":"ContainerStarted","Data":"9e4e475d32cc67da9ae5ddf1b7985ebbd0b0962088816e0923d294d274d59256"} Jul 23 09:17:55.700287 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:55.699932 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cvxp6" Jul 23 09:17:55.720321 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:55.720242 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cvxp6" podStartSLOduration=16.84672896 podStartE2EDuration="19.720221312s" totalImagesPullingTime="2.873492352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:17:55.719125864 +0000 UTC m=+55.929309016" watchObservedRunningTime="2026-07-23 09:17:55.720221312 +0000 UTC m=+55.930404463" Jul 23 09:17:58.713549 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:17:58.713511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kj2j4" event={"ID":"801082ef-11db-4f35-86fc-3806e65015cf","Type":"ContainerStarted","Data":"e02f06a5f83ff1b026a6032e23133c99a734be271f5fd697be2a88c8d6285398"} Jul 23 09:18:01.614820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:01.614791 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4rwpq" Jul 23 09:18:02.729624 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.729587 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kj2j4" event={"ID":"801082ef-11db-4f35-86fc-3806e65015cf","Type":"ContainerStarted","Data":"a2cb53c39a11328889f17cdf47e257a3edc12645bc13f2f97135808cad71e7c5"} Jul 23 09:18:02.766436 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.766398 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76"] Jul 23 09:18:02.788937 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.788905 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7"] Jul 23 09:18:02.789101 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.789048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.793614 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.793590 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 09:18:02.794150 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.794129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 09:18:02.794938 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.794915 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 09:18:02.795037 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.794931 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 09:18:02.802712 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.802687 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76"] Jul 23 09:18:02.802829 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.802734 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7"] Jul 23 09:18:02.802829 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.802805 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.814026 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.814006 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 23 09:18:02.820309 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.820281 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 23 09:18:02.820451 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.820314 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 23 09:18:02.820599 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.820583 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 23 09:18:02.860738 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.860738 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc4343-77aa-408e-8e92-211c63adab78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.860980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v65mp\" (UniqueName: \"kubernetes.io/projected/5bfc4343-77aa-408e-8e92-211c63adab78-kube-api-access-v65mp\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.860980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/43a367a8-31ea-45ac-93c3-c5337962ac68-klusterlet-config\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.860980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.860980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.860980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860961 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43a367a8-31ea-45ac-93c3-c5337962ac68-tmp\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.861235 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.860990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr9qk\" (UniqueName: \"kubernetes.io/projected/43a367a8-31ea-45ac-93c3-c5337962ac68-kube-api-access-qr9qk\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.861235 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.861014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962218 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr9qk\" (UniqueName: \"kubernetes.io/projected/43a367a8-31ea-45ac-93c3-c5337962ac68-kube-api-access-qr9qk\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962226 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc4343-77aa-408e-8e92-211c63adab78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v65mp\" (UniqueName: \"kubernetes.io/projected/5bfc4343-77aa-408e-8e92-211c63adab78-kube-api-access-v65mp\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/43a367a8-31ea-45ac-93c3-c5337962ac68-klusterlet-config\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.962459 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.962818 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.962505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.963200 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.963021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43a367a8-31ea-45ac-93c3-c5337962ac68-tmp\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.963504 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.963406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bfc4343-77aa-408e-8e92-211c63adab78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.963664 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.963584 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43a367a8-31ea-45ac-93c3-c5337962ac68-tmp\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.965924 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.965558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.965924 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.965880 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.966080 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.965990 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.966779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.966760 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5bfc4343-77aa-408e-8e92-211c63adab78-ca\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:02.966895 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.966877 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/43a367a8-31ea-45ac-93c3-c5337962ac68-klusterlet-config\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:02.981689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:02.981616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr9qk\" (UniqueName: \"kubernetes.io/projected/43a367a8-31ea-45ac-93c3-c5337962ac68-kube-api-access-qr9qk\") pod \"klusterlet-addon-workmgr-864487ccf4-slb76\" (UID: \"43a367a8-31ea-45ac-93c3-c5337962ac68\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:03.001685 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.001655 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v65mp\" (UniqueName: \"kubernetes.io/projected/5bfc4343-77aa-408e-8e92-211c63adab78-kube-api-access-v65mp\") pod \"cluster-proxy-proxy-agent-64b69b85cb-sb7g7\" (UID: \"5bfc4343-77aa-408e-8e92-211c63adab78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:03.099048 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.099010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:03.118111 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.118079 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" Jul 23 09:18:03.272988 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.272914 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76"] Jul 23 09:18:03.277367 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:03.277306 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a367a8_31ea_45ac_93c3_c5337962ac68.slice/crio-7c869ffd3bcdcb1493b964912b070f5e8a38fc4e78fb9ff05297aef7a5ffb9c3 WatchSource:0}: Error finding container 7c869ffd3bcdcb1493b964912b070f5e8a38fc4e78fb9ff05297aef7a5ffb9c3: Status 404 returned error can't find the container with id 7c869ffd3bcdcb1493b964912b070f5e8a38fc4e78fb9ff05297aef7a5ffb9c3 Jul 23 09:18:03.285637 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.285611 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7"] Jul 23 09:18:03.289226 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:03.288773 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bfc4343_77aa_408e_8e92_211c63adab78.slice/crio-832c07ec6885de8f2fa7b43cf8aec30baafbbbebd0e913402107de869ce9abec WatchSource:0}: Error finding container 832c07ec6885de8f2fa7b43cf8aec30baafbbbebd0e913402107de869ce9abec: Status 404 returned error can't find the container with id 832c07ec6885de8f2fa7b43cf8aec30baafbbbebd0e913402107de869ce9abec Jul 23 09:18:03.734091 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.734047 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" event={"ID":"5bfc4343-77aa-408e-8e92-211c63adab78","Type":"ContainerStarted","Data":"832c07ec6885de8f2fa7b43cf8aec30baafbbbebd0e913402107de869ce9abec"} Jul 23 09:18:03.735392 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:03.735365 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" event={"ID":"43a367a8-31ea-45ac-93c3-c5337962ac68","Type":"ContainerStarted","Data":"7c869ffd3bcdcb1493b964912b070f5e8a38fc4e78fb9ff05297aef7a5ffb9c3"} Jul 23 09:18:04.983456 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:04.983017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:18:04.988130 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:04.987868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 09:18:05.001217 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.001107 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1278b23a-b894-4fa2-8eef-497ee694573b-original-pull-secret\") pod \"global-pull-secret-syncer-jxtrx\" (UID: \"1278b23a-b894-4fa2-8eef-497ee694573b\") " pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:18:05.083663 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.083623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:18:05.086823 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.086767 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 09:18:05.099386 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.099303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ee8dcd-7f5f-4fbd-82f6-14536e167955-metrics-certs\") pod \"network-metrics-daemon-nmd8t\" (UID: \"92ee8dcd-7f5f-4fbd-82f6-14536e167955\") " pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:18:05.160749 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.160713 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jxtrx" Jul 23 09:18:05.184285 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.184241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:18:05.190808 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.190773 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 09:18:05.199300 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.199265 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 09:18:05.210098 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.210068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcsmt\" (UniqueName: \"kubernetes.io/projected/ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09-kube-api-access-zcsmt\") pod \"network-check-target-zxz2t\" (UID: \"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09\") " pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:18:05.256207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.255907 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s2sdc\"" Jul 23 09:18:05.262918 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.262680 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tghcj\"" Jul 23 09:18:05.264265 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.264002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:18:05.269490 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.269464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nmd8t" Jul 23 09:18:05.707962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:05.707220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cvxp6" Jul 23 09:18:10.636981 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.636957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zxz2t"] Jul 23 09:18:10.640865 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:10.640835 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6cab58_0cc7_4716_bbe3_6a8b87c2ce09.slice/crio-714384f3d32a0d0081dc7db447b6edc42f5c3dd53a55c7f80ca77c44021cb6ca WatchSource:0}: Error finding container 714384f3d32a0d0081dc7db447b6edc42f5c3dd53a55c7f80ca77c44021cb6ca: Status 404 returned error can't find the container with id 714384f3d32a0d0081dc7db447b6edc42f5c3dd53a55c7f80ca77c44021cb6ca Jul 23 09:18:10.667629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.667164 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jxtrx"] Jul 23 09:18:10.667629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.667609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nmd8t"] Jul 23 09:18:10.671034 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:10.670984 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ee8dcd_7f5f_4fbd_82f6_14536e167955.slice/crio-01895f6c0a4a63b2ddf38230f0622f3d1e25810171fd1d398f822f8999f99c11 WatchSource:0}: Error finding container 01895f6c0a4a63b2ddf38230f0622f3d1e25810171fd1d398f822f8999f99c11: Status 404 returned error can't find the container with id 01895f6c0a4a63b2ddf38230f0622f3d1e25810171fd1d398f822f8999f99c11 Jul 23 09:18:10.671667 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:10.671638 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1278b23a_b894_4fa2_8eef_497ee694573b.slice/crio-696a0ccb4b2948762bec98bb5d13225c5b11b1a9151d6f01b22cc74b679174ab WatchSource:0}: Error finding container 696a0ccb4b2948762bec98bb5d13225c5b11b1a9151d6f01b22cc74b679174ab: Status 404 returned error can't find the container with id 696a0ccb4b2948762bec98bb5d13225c5b11b1a9151d6f01b22cc74b679174ab Jul 23 09:18:10.764192 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.764154 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kj2j4" event={"ID":"801082ef-11db-4f35-86fc-3806e65015cf","Type":"ContainerStarted","Data":"429dab287dd928883fac04b075d7eac8c191164632df1592c4604029682079ad"} Jul 23 09:18:10.765696 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.765669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" event={"ID":"5bfc4343-77aa-408e-8e92-211c63adab78","Type":"ContainerStarted","Data":"f2e94abd01fd635bf56d3d8c5fbf42c3bd51204f03fddaed02712320138a701d"} Jul 23 09:18:10.766727 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.766699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zxz2t" event={"ID":"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09","Type":"ContainerStarted","Data":"714384f3d32a0d0081dc7db447b6edc42f5c3dd53a55c7f80ca77c44021cb6ca"} Jul 23 09:18:10.768070 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.768045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" event={"ID":"43a367a8-31ea-45ac-93c3-c5337962ac68","Type":"ContainerStarted","Data":"cfcca58fb6071acd1f353ecf4ba8ad52a19ec60d6c4ea6aa832d04ffe1443057"} Jul 23 09:18:10.768235 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.768212 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:10.769750 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.769689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nmd8t" event={"ID":"92ee8dcd-7f5f-4fbd-82f6-14536e167955","Type":"ContainerStarted","Data":"01895f6c0a4a63b2ddf38230f0622f3d1e25810171fd1d398f822f8999f99c11"} Jul 23 09:18:10.770421 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.770346 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" Jul 23 09:18:10.771371 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.771316 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jxtrx" event={"ID":"1278b23a-b894-4fa2-8eef-497ee694573b","Type":"ContainerStarted","Data":"696a0ccb4b2948762bec98bb5d13225c5b11b1a9151d6f01b22cc74b679174ab"} Jul 23 09:18:10.816016 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.815966 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kj2j4" podStartSLOduration=16.768466842 podStartE2EDuration="33.815949064s" totalImagesPullingTime="17.047482222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:10.789418155 +0000 UTC m=+70.999601306" watchObservedRunningTime="2026-07-23 09:18:10.815949064 +0000 UTC m=+71.026132214" Jul 23 09:18:10.816628 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:10.816598 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864487ccf4-slb76" podStartSLOduration=1.6240314310000001 podStartE2EDuration="8.816589482s" totalImagesPullingTime="7.192558051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:10.815486443 +0000 UTC m=+71.025669594" watchObservedRunningTime="2026-07-23 09:18:10.816589482 +0000 UTC m=+71.026772677" Jul 23 09:18:11.778045 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:11.777963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" event={"ID":"5bfc4343-77aa-408e-8e92-211c63adab78","Type":"ContainerStarted","Data":"26e7489ab5bed2befaa7083b91d4de25bd03d3ce27516c243390beedc5958ae3"} Jul 23 09:18:11.778045 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:11.778015 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" event={"ID":"5bfc4343-77aa-408e-8e92-211c63adab78","Type":"ContainerStarted","Data":"6d0c4cfccd44d937c4365c850cd44399c62f362502025dbd3a8dd9397e955481"} Jul 23 09:18:11.804221 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:11.804081 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64b69b85cb-sb7g7" podStartSLOduration=2.606262129 podStartE2EDuration="9.804058334s" totalImagesPullingTime="7.197796205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:11.803695732 +0000 UTC m=+72.013878881" watchObservedRunningTime="2026-07-23 09:18:11.804058334 +0000 UTC m=+72.014241483" Jul 23 09:18:12.784566 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:12.784471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nmd8t" event={"ID":"92ee8dcd-7f5f-4fbd-82f6-14536e167955","Type":"ContainerStarted","Data":"beaa018ca4e9b553b0d2feed65f03fa888373499db2247086ccb041eff5230ee"} Jul 23 09:18:15.798569 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.798531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zxz2t" event={"ID":"ec6cab58-0cc7-4716-bbe3-6a8b87c2ce09","Type":"ContainerStarted","Data":"8c19920063ae160c70dc65be4cdc114e9c3cf3f0d7c46a25ed187b61a29346dd"} Jul 23 09:18:15.799065 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.798637 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:18:15.800399 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.800369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nmd8t" event={"ID":"92ee8dcd-7f5f-4fbd-82f6-14536e167955","Type":"ContainerStarted","Data":"3f57370a32e18830bde714bf03305b6335cad9aa669709b40180259104c595ab"} Jul 23 09:18:15.801685 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.801664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jxtrx" event={"ID":"1278b23a-b894-4fa2-8eef-497ee694573b","Type":"ContainerStarted","Data":"b3e1685a9f9f96f0e69a4caf1aae341de7103e455ee6401f223759fbfbd7e4a6"} Jul 23 09:18:15.818389 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.818322 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zxz2t" podStartSLOduration=70.305832012 podStartE2EDuration="1m14.818307683s" totalImagesPullingTime="4.512475671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:15.817921303 +0000 UTC m=+76.028104455" watchObservedRunningTime="2026-07-23 09:18:15.818307683 +0000 UTC m=+76.028490811" Jul 23 09:18:15.842073 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.842016 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nmd8t" podStartSLOduration=72.994384008 podStartE2EDuration="1m14.841997969s" totalImagesPullingTime="1.847613961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:15.841537625 +0000 UTC m=+76.051720774" watchObservedRunningTime="2026-07-23 09:18:15.841997969 +0000 UTC m=+76.052181118" Jul 23 09:18:15.861713 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:15.861660 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jxtrx" podStartSLOduration=70.377191054 podStartE2EDuration="1m14.861639136s" totalImagesPullingTime="4.484448082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 09:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:15.860561481 +0000 UTC m=+76.070744630" watchObservedRunningTime="2026-07-23 09:18:15.861639136 +0000 UTC m=+76.071822285" Jul 23 09:18:16.039408 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.039368 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6wwzm"] Jul 23 09:18:16.042162 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.042141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.045115 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.045086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 09:18:16.045266 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.045247 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dcjpd\"" Jul 23 09:18:16.045638 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.045614 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 09:18:16.045977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.045769 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 09:18:16.046390 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.046370 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 09:18:16.046501 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.046372 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 09:18:16.047464 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.047441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 09:18:16.172095 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-root\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172095 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-metrics-client-ca\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-textfile\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-tls\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b887c\" (UniqueName: \"kubernetes.io/projected/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-kube-api-access-b887c\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-wtmp\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.172380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.172299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-sys\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273614 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b887c\" (UniqueName: \"kubernetes.io/projected/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-kube-api-access-b887c\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-wtmp\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-sys\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-root\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-metrics-client-ca\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273825 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273820 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-wtmp\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-textfile\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273837 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-sys\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.273912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-tls\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.274360 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.273838 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-root\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.274360 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.274208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-textfile\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.274551 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.274525 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.274551 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.274526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-metrics-client-ca\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.276895 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.276872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.276972 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.276934 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-node-exporter-tls\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.283441 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.283416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b887c\" (UniqueName: \"kubernetes.io/projected/b3ddbb4e-b08a-49c2-ad06-62b3be3316c1-kube-api-access-b887c\") pod \"node-exporter-6wwzm\" (UID: \"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1\") " pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.354357 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.354258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wwzm" Jul 23 09:18:16.365593 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:18:16.365566 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ddbb4e_b08a_49c2_ad06_62b3be3316c1.slice/crio-e84ae1bb1cdf70e679f20fcf94a0bc7cd93cd158d7d6a1bd48c87f7780b653eb WatchSource:0}: Error finding container e84ae1bb1cdf70e679f20fcf94a0bc7cd93cd158d7d6a1bd48c87f7780b653eb: Status 404 returned error can't find the container with id e84ae1bb1cdf70e679f20fcf94a0bc7cd93cd158d7d6a1bd48c87f7780b653eb Jul 23 09:18:16.809485 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:16.809449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwzm" event={"ID":"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1","Type":"ContainerStarted","Data":"e84ae1bb1cdf70e679f20fcf94a0bc7cd93cd158d7d6a1bd48c87f7780b653eb"} Jul 23 09:18:17.814603 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:17.814564 2562 generic.go:356] "Generic (PLEG): container finished" podID="b3ddbb4e-b08a-49c2-ad06-62b3be3316c1" containerID="f86d648edca2a92a829554598c5b493532b6922cdd50ee3f14dccfd129b39a63" exitCode=0 Jul 23 09:18:17.814984 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:17.814638 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwzm" event={"ID":"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1","Type":"ContainerDied","Data":"f86d648edca2a92a829554598c5b493532b6922cdd50ee3f14dccfd129b39a63"} Jul 23 09:18:18.821010 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:18.820967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwzm" event={"ID":"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1","Type":"ContainerStarted","Data":"073a30a7ed0f731207d8d274b87ffdee563733060c3221a8560f515fcf1c6fa6"} Jul 23 09:18:18.821010 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:18.821020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwzm" event={"ID":"b3ddbb4e-b08a-49c2-ad06-62b3be3316c1","Type":"ContainerStarted","Data":"b48e70058fba8a7cee11cc0b4657dda322e784d0d9048978b6b33a02cc0c9fcb"} Jul 23 09:18:18.843010 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:18.842966 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6wwzm" podStartSLOduration=1.849900999 podStartE2EDuration="2.842950742s" totalImagesPullingTime="921.774015ms" totalInitContainerRuntime="71.275728ms" isStatefulPod=true podCreationTimestamp="2026-07-23 09:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:18:18.841126147 +0000 UTC m=+79.051309301" watchObservedRunningTime="2026-07-23 09:18:18.842950742 +0000 UTC m=+79.053133892" Jul 23 09:18:46.812442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:18:46.812408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zxz2t" Jul 23 09:21:34.619618 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.619531 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf"] Jul 23 09:21:34.621870 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.621848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.626186 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.626165 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-h624w\"" Jul 23 09:21:34.626295 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.626171 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 09:21:34.626774 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.626721 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 09:21:34.637675 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.637655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf"] Jul 23 09:21:34.785977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.785933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beaa8acc-e372-42bf-89b2-f9204e38903a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.786140 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.786000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn4w2\" (UniqueName: \"kubernetes.io/projected/beaa8acc-e372-42bf-89b2-f9204e38903a-kube-api-access-hn4w2\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.887306 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.887230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4w2\" (UniqueName: \"kubernetes.io/projected/beaa8acc-e372-42bf-89b2-f9204e38903a-kube-api-access-hn4w2\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.887306 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.887282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beaa8acc-e372-42bf-89b2-f9204e38903a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.887644 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.887629 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beaa8acc-e372-42bf-89b2-f9204e38903a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.901413 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.901389 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn4w2\" (UniqueName: \"kubernetes.io/projected/beaa8acc-e372-42bf-89b2-f9204e38903a-kube-api-access-hn4w2\") pod \"openshift-lws-operator-fd8ccff4c-sxnkf\" (UID: \"beaa8acc-e372-42bf-89b2-f9204e38903a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:34.929888 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:34.929848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" Jul 23 09:21:35.058039 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:35.058010 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf"] Jul 23 09:21:35.060513 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:21:35.060490 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeaa8acc_e372_42bf_89b2_f9204e38903a.slice/crio-9f247f208e0f6f5c5528cbb4ec1054032bffae9f5e3e60b153d83e0987b67312 WatchSource:0}: Error finding container 9f247f208e0f6f5c5528cbb4ec1054032bffae9f5e3e60b153d83e0987b67312: Status 404 returned error can't find the container with id 9f247f208e0f6f5c5528cbb4ec1054032bffae9f5e3e60b153d83e0987b67312 Jul 23 09:21:35.459928 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:35.459893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" event={"ID":"beaa8acc-e372-42bf-89b2-f9204e38903a","Type":"ContainerStarted","Data":"9f247f208e0f6f5c5528cbb4ec1054032bffae9f5e3e60b153d83e0987b67312"} Jul 23 09:21:38.472115 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:38.472080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" event={"ID":"beaa8acc-e372-42bf-89b2-f9204e38903a","Type":"ContainerStarted","Data":"2c70febbc9557e243589d0811335c8305ad54da3098e96e4bbc2ad44968de507"} Jul 23 09:21:38.490849 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:38.490807 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sxnkf" podStartSLOduration=1.876494357 podStartE2EDuration="4.490792179s" totalImagesPullingTime="2.614297822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:21:38.489428254 +0000 UTC m=+278.699611404" watchObservedRunningTime="2026-07-23 09:21:38.490792179 +0000 UTC m=+278.700975328" Jul 23 09:21:54.132921 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.132888 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c"] Jul 23 09:21:54.137950 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.137926 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.141907 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.141878 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 09:21:54.142442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.142424 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lfn6j\"" Jul 23 09:21:54.142525 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.142449 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 09:21:54.142729 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.142713 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 09:21:54.142796 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.142725 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 09:21:54.181804 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.181772 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c"] Jul 23 09:21:54.332615 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.332576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ssm\" (UniqueName: \"kubernetes.io/projected/718ba171-fc11-4ef7-98bc-774a0d45ebc5-kube-api-access-g2ssm\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.332615 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.332613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-manifests\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.332826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.332688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.332826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.332722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-tmp\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.332826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.332770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.433628 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.433589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.433628 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.433632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ssm\" (UniqueName: \"kubernetes.io/projected/718ba171-fc11-4ef7-98bc-774a0d45ebc5-kube-api-access-g2ssm\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.433847 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.433650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-manifests\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.433847 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.433675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.433847 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.433711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-tmp\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.434085 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.434052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-manifests\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.436361 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.436320 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718ba171-fc11-4ef7-98bc-774a0d45ebc5-tmp\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.436555 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.436539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.436606 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.436583 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718ba171-fc11-4ef7-98bc-774a0d45ebc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.441702 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.441680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ssm\" (UniqueName: \"kubernetes.io/projected/718ba171-fc11-4ef7-98bc-774a0d45ebc5-kube-api-access-g2ssm\") pod \"opendatahub-operator-controller-manager-64d6699bc9-nb66c\" (UID: \"718ba171-fc11-4ef7-98bc-774a0d45ebc5\") " pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.447537 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.447516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:21:54.577900 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:54.577868 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c"] Jul 23 09:21:54.581419 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:21:54.581393 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod718ba171_fc11_4ef7_98bc_774a0d45ebc5.slice/crio-caf5e2d34b29243a5e68e21dd44bef770a721b481135fcc1d0bc8a3ec934d3e9 WatchSource:0}: Error finding container caf5e2d34b29243a5e68e21dd44bef770a721b481135fcc1d0bc8a3ec934d3e9: Status 404 returned error can't find the container with id caf5e2d34b29243a5e68e21dd44bef770a721b481135fcc1d0bc8a3ec934d3e9 Jul 23 09:21:55.534230 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:55.534189 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" event={"ID":"718ba171-fc11-4ef7-98bc-774a0d45ebc5","Type":"ContainerStarted","Data":"caf5e2d34b29243a5e68e21dd44bef770a721b481135fcc1d0bc8a3ec934d3e9"} Jul 23 09:21:58.546657 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:58.546618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" event={"ID":"718ba171-fc11-4ef7-98bc-774a0d45ebc5","Type":"ContainerStarted","Data":"e3e8b984822d39aaf4f644509e082a274cb9934141275e7da73a40305bae6071"} Jul 23 09:21:59.551601 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:59.551570 2562 generic.go:356] "Generic (PLEG): container finished" podID="718ba171-fc11-4ef7-98bc-774a0d45ebc5" containerID="e3e8b984822d39aaf4f644509e082a274cb9934141275e7da73a40305bae6071" exitCode=0 Jul 23 09:21:59.551996 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:21:59.551657 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" event={"ID":"718ba171-fc11-4ef7-98bc-774a0d45ebc5","Type":"ContainerDied","Data":"e3e8b984822d39aaf4f644509e082a274cb9934141275e7da73a40305bae6071"} Jul 23 09:22:00.170994 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.170958 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:22:00.174120 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.174096 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 09:22:00.177175 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.177153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:22:00.557370 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.557311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" event={"ID":"718ba171-fc11-4ef7-98bc-774a0d45ebc5","Type":"ContainerStarted","Data":"07f37a9052154267f72c6b6224b5a95000b7fa10469e6eb024cc463f9fd8d620"} Jul 23 09:22:00.559584 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.557498 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:22:00.586806 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:00.586755 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" podStartSLOduration=2.134283126 podStartE2EDuration="6.586739146s" totalImagesPullingTime="3.406322745s" totalInitContainerRuntime="1.046133275s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:21:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:00.584657728 +0000 UTC m=+300.794840878" watchObservedRunningTime="2026-07-23 09:22:00.586739146 +0000 UTC m=+300.796922296" Jul 23 09:22:11.563926 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:11.563886 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64d6699bc9-nb66c" Jul 23 09:22:13.935508 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.935476 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pr4jf"] Jul 23 09:22:13.943237 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.943216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:13.948679 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.948659 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5m646\"" Jul 23 09:22:13.948781 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.948704 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 09:22:13.962072 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.962030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pr4jf"] Jul 23 09:22:13.973011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.972987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:13.973152 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:13.973026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28wc7\" (UniqueName: \"kubernetes.io/projected/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-kube-api-access-28wc7\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.074454 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.074415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.074645 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.074469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28wc7\" (UniqueName: \"kubernetes.io/projected/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-kube-api-access-28wc7\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.074645 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:14.074587 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 09:22:14.074755 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:14.074666 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert podName:075a3e0d-dc49-43fa-ad2f-16aedc544bb8 nodeName:}" failed. No retries permitted until 2026-07-23 09:22:14.574644143 +0000 UTC m=+314.784827284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert") pod "odh-model-controller-7998c67dc4-pr4jf" (UID: "075a3e0d-dc49-43fa-ad2f-16aedc544bb8") : secret "odh-model-controller-webhook-cert" not found Jul 23 09:22:14.086729 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.086697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28wc7\" (UniqueName: \"kubernetes.io/projected/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-kube-api-access-28wc7\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.578485 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.578449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.581094 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.581065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/075a3e0d-dc49-43fa-ad2f-16aedc544bb8-cert\") pod \"odh-model-controller-7998c67dc4-pr4jf\" (UID: \"075a3e0d-dc49-43fa-ad2f-16aedc544bb8\") " pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.853256 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.853156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:14.978748 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.978698 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pr4jf"] Jul 23 09:22:14.983113 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:14.983086 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075a3e0d_dc49_43fa_ad2f_16aedc544bb8.slice/crio-0228408394da1471e8f8f46f086767c65be6ae785e577e3865510d92951ec1a1 WatchSource:0}: Error finding container 0228408394da1471e8f8f46f086767c65be6ae785e577e3865510d92951ec1a1: Status 404 returned error can't find the container with id 0228408394da1471e8f8f46f086767c65be6ae785e577e3865510d92951ec1a1 Jul 23 09:22:14.984426 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:14.984406 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 09:22:15.610981 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:15.610942 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" event={"ID":"075a3e0d-dc49-43fa-ad2f-16aedc544bb8","Type":"ContainerStarted","Data":"0228408394da1471e8f8f46f086767c65be6ae785e577e3865510d92951ec1a1"} Jul 23 09:22:18.624085 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:18.624051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" event={"ID":"075a3e0d-dc49-43fa-ad2f-16aedc544bb8","Type":"ContainerStarted","Data":"bbb7d0e0708a0e9a32f5ddcc9df6d2c6fa01c3715e249de238ac372e65bbce83"} Jul 23 09:22:18.624475 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:18.624166 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:18.641359 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:18.641286 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" podStartSLOduration=2.785718385 podStartE2EDuration="5.641270735s" totalImagesPullingTime="2.85555235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:18.640114084 +0000 UTC m=+318.850297228" watchObservedRunningTime="2026-07-23 09:22:18.641270735 +0000 UTC m=+318.851453937" Jul 23 09:22:19.224483 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.224448 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9"] Jul 23 09:22:19.227570 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.227549 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.229820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.229792 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 09:22:19.229936 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.229843 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 09:22:19.229936 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.229871 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nktwx\"" Jul 23 09:22:19.230614 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.230598 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 09:22:19.230737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.230625 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 09:22:19.238862 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.238841 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9"] Jul 23 09:22:19.318648 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.318609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.318816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.318663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66a53353-2dc0-48ca-b973-82d6d050f1cb-tmp\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.318816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.318771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7fcl\" (UniqueName: \"kubernetes.io/projected/66a53353-2dc0-48ca-b973-82d6d050f1cb-kube-api-access-t7fcl\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.419442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.419410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7fcl\" (UniqueName: \"kubernetes.io/projected/66a53353-2dc0-48ca-b973-82d6d050f1cb-kube-api-access-t7fcl\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.419604 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.419483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.419604 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.419532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66a53353-2dc0-48ca-b973-82d6d050f1cb-tmp\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.419683 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:19.419618 2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 23 09:22:19.419720 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:19.419702 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs podName:66a53353-2dc0-48ca-b973-82d6d050f1cb nodeName:}" failed. No retries permitted until 2026-07-23 09:22:19.919681878 +0000 UTC m=+320.129865006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs") pod "kube-auth-proxy-5dccdf9c4-8bfm9" (UID: "66a53353-2dc0-48ca-b973-82d6d050f1cb") : secret "kube-auth-proxy-tls" not found Jul 23 09:22:19.421885 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.421863 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66a53353-2dc0-48ca-b973-82d6d050f1cb-tmp\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.430006 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.429979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7fcl\" (UniqueName: \"kubernetes.io/projected/66a53353-2dc0-48ca-b973-82d6d050f1cb-kube-api-access-t7fcl\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.924477 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.924439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:19.927057 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:19.927035 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a53353-2dc0-48ca-b973-82d6d050f1cb-tls-certs\") pod \"kube-auth-proxy-5dccdf9c4-8bfm9\" (UID: \"66a53353-2dc0-48ca-b973-82d6d050f1cb\") " pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:20.136956 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:20.136922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" Jul 23 09:22:20.267537 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:20.267512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9"] Jul 23 09:22:20.269225 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:20.269201 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a53353_2dc0_48ca_b973_82d6d050f1cb.slice/crio-7f86e7bb009799d8ca978d0aeb02028d81a15a031b094d3869daa306865e1a73 WatchSource:0}: Error finding container 7f86e7bb009799d8ca978d0aeb02028d81a15a031b094d3869daa306865e1a73: Status 404 returned error can't find the container with id 7f86e7bb009799d8ca978d0aeb02028d81a15a031b094d3869daa306865e1a73 Jul 23 09:22:20.634439 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:20.634357 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" event={"ID":"66a53353-2dc0-48ca-b973-82d6d050f1cb","Type":"ContainerStarted","Data":"7f86e7bb009799d8ca978d0aeb02028d81a15a031b094d3869daa306865e1a73"} Jul 23 09:22:21.895592 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.895559 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq"] Jul 23 09:22:21.898660 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.898629 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.902069 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.902040 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 09:22:21.902478 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.902272 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 09:22:21.902478 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.902290 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 09:22:21.902478 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.902357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9f95b\"" Jul 23 09:22:21.902478 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.902358 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 09:22:21.917361 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.917313 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq"] Jul 23 09:22:21.939771 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.939931 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.939931 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6jbt\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-kube-api-access-w6jbt\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.939931 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.939931 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.940125 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:21.940125 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:21.939991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b00f2c3c-20bb-483a-9d51-624bd6f4990f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.040723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.040678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.040723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.040729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.040964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.040860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.040964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.040909 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b00f2c3c-20bb-483a-9d51-624bd6f4990f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.041076 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.040971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.041076 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.041032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.041076 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.041066 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6jbt\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-kube-api-access-w6jbt\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.041644 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.041592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.044031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.043650 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.044031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.043982 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.044205 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.044048 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b00f2c3c-20bb-483a-9d51-624bd6f4990f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.044205 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.044082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.051186 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.051161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.051872 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.051854 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6jbt\" (UniqueName: \"kubernetes.io/projected/b00f2c3c-20bb-483a-9d51-624bd6f4990f-kube-api-access-w6jbt\") pod \"istiod-openshift-gateway-94bb8fbfd-qzpsq\" (UID: \"b00f2c3c-20bb-483a-9d51-624bd6f4990f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.210511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.210480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:22.827882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:22.827817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq"] Jul 23 09:22:22.831787 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:22.831764 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00f2c3c_20bb_483a_9d51_624bd6f4990f.slice/crio-bab48bfc98d8f67ad095cb23369d8e927efd72a293115818878ef7ae897f4492 WatchSource:0}: Error finding container bab48bfc98d8f67ad095cb23369d8e927efd72a293115818878ef7ae897f4492: Status 404 returned error can't find the container with id bab48bfc98d8f67ad095cb23369d8e927efd72a293115818878ef7ae897f4492 Jul 23 09:22:23.647471 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:23.647432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" event={"ID":"66a53353-2dc0-48ca-b973-82d6d050f1cb","Type":"ContainerStarted","Data":"2a11c11c0714e4eb1c3febd8672114e3a0a3a3ab04876bcd9ddb98a344bc9fdd"} Jul 23 09:22:23.648643 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:23.648619 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" event={"ID":"b00f2c3c-20bb-483a-9d51-624bd6f4990f","Type":"ContainerStarted","Data":"bab48bfc98d8f67ad095cb23369d8e927efd72a293115818878ef7ae897f4492"} Jul 23 09:22:23.693127 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:23.693073 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dccdf9c4-8bfm9" podStartSLOduration=2.233990427 podStartE2EDuration="4.693056017s" totalImagesPullingTime="2.45906559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:23.691108119 +0000 UTC m=+323.901291269" watchObservedRunningTime="2026-07-23 09:22:23.693056017 +0000 UTC m=+323.903239166" Jul 23 09:22:24.703609 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.703571 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9"] Jul 23 09:22:24.706966 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.706944 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.710475 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.710226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 09:22:24.710475 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.710367 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rrm5z\"" Jul 23 09:22:24.734368 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.733303 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9"] Jul 23 09:22:24.734368 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.733384 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn"] Jul 23 09:22:24.736791 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.736770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:24.740493 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.740378 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6g79l\"" Jul 23 09:22:24.740937 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.740747 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 09:22:24.751752 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.751730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn"] Jul 23 09:22:24.764972 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.764943 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.765104 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.764990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75qwx\" (UniqueName: \"kubernetes.io/projected/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-kube-api-access-75qwx\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.866491 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.866451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg6hw\" (UniqueName: \"kubernetes.io/projected/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-kube-api-access-zg6hw\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:24.866622 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.866526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:24.866622 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.866562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.866622 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.866596 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75qwx\" (UniqueName: \"kubernetes.io/projected/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-kube-api-access-75qwx\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.866732 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:24.866705 2562 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 23 09:22:24.866804 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:24.866792 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert podName:936103be-6c95-4ee3-a6ee-0abd6aa2ebe2 nodeName:}" failed. No retries permitted until 2026-07-23 09:22:25.366771645 +0000 UTC m=+325.576954778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert") pod "kserve-controller-manager-d6f4b65b8-kzdd9" (UID: "936103be-6c95-4ee3-a6ee-0abd6aa2ebe2") : secret "kserve-webhook-server-cert" not found Jul 23 09:22:24.889145 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.889111 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75qwx\" (UniqueName: \"kubernetes.io/projected/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-kube-api-access-75qwx\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:24.967205 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.967087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg6hw\" (UniqueName: \"kubernetes.io/projected/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-kube-api-access-zg6hw\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:24.967205 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.967178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:24.967461 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:24.967285 2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 23 09:22:24.967461 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:22:24.967377 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert podName:b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b nodeName:}" failed. No retries permitted until 2026-07-23 09:22:25.467354929 +0000 UTC m=+325.677538076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert") pod "llmisvc-controller-manager-777c6bd7d5-hbdvn" (UID: "b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b") : secret "llmisvc-webhook-server-cert" not found Jul 23 09:22:24.985064 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:24.985034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg6hw\" (UniqueName: \"kubernetes.io/projected/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-kube-api-access-zg6hw\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:25.371677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.371589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:25.374349 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.374292 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/936103be-6c95-4ee3-a6ee-0abd6aa2ebe2-cert\") pod \"kserve-controller-manager-d6f4b65b8-kzdd9\" (UID: \"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:25.472391 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.472343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:25.475182 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.475158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hbdvn\" (UID: \"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:25.620428 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.620390 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:25.649840 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.649807 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:25.831398 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.831265 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:22:25.832417 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.832132 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:22:25.927379 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.927307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn"] Jul 23 09:22:25.929648 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:25.929618 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5c50541_6b3f_4fa0_99fa_f6671d6a8c5b.slice/crio-8c364113d24ac4f47be2945cd5431ef4e41a3fee970eb8a16c99241a92f2c421 WatchSource:0}: Error finding container 8c364113d24ac4f47be2945cd5431ef4e41a3fee970eb8a16c99241a92f2c421: Status 404 returned error can't find the container with id 8c364113d24ac4f47be2945cd5431ef4e41a3fee970eb8a16c99241a92f2c421 Jul 23 09:22:25.968238 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:25.968202 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9"] Jul 23 09:22:25.971609 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:25.971571 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936103be_6c95_4ee3_a6ee_0abd6aa2ebe2.slice/crio-cf696c413c1a648ac472d7359c607a910bcef8a7d0a31382d0b73e97002d970c WatchSource:0}: Error finding container cf696c413c1a648ac472d7359c607a910bcef8a7d0a31382d0b73e97002d970c: Status 404 returned error can't find the container with id cf696c413c1a648ac472d7359c607a910bcef8a7d0a31382d0b73e97002d970c Jul 23 09:22:26.177259 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.177225 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh"] Jul 23 09:22:26.180548 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.180524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.186972 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.186948 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-g9dzk\"" Jul 23 09:22:26.223212 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.223181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh"] Jul 23 09:22:26.280530 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c15706b-bbba-4497-8153-db9f3dbe9de9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.280737 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280735 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.281062 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.281062 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.280837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl7w9\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-kube-api-access-sl7w9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.381950 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.381914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.381950 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.381959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.381982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c15706b-bbba-4497-8153-db9f3dbe9de9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382454 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7w9\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-kube-api-access-sl7w9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382454 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382454 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382447 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382616 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382616 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382587 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.382990 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.382963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.383104 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.383046 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.383167 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.383145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c15706b-bbba-4497-8153-db9f3dbe9de9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.384961 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.384935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.385148 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.385124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.397432 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.397366 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.397640 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.397613 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7w9\" (UniqueName: \"kubernetes.io/projected/9c15706b-bbba-4497-8153-db9f3dbe9de9-kube-api-access-sl7w9\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bt5dh\" (UID: \"9c15706b-bbba-4497-8153-db9f3dbe9de9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.490668 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.490553 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:26.649956 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.649890 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh"] Jul 23 09:22:26.652416 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:22:26.652384 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c15706b_bbba_4497_8153_db9f3dbe9de9.slice/crio-0cce0e9f7d29cac12ba11d0983c21799f46ca41bf3d8c84f7bb9ca71daa4a4fb WatchSource:0}: Error finding container 0cce0e9f7d29cac12ba11d0983c21799f46ca41bf3d8c84f7bb9ca71daa4a4fb: Status 404 returned error can't find the container with id 0cce0e9f7d29cac12ba11d0983c21799f46ca41bf3d8c84f7bb9ca71daa4a4fb Jul 23 09:22:26.663743 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.663708 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" event={"ID":"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b","Type":"ContainerStarted","Data":"8c364113d24ac4f47be2945cd5431ef4e41a3fee970eb8a16c99241a92f2c421"} Jul 23 09:22:26.665163 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.665132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" event={"ID":"9c15706b-bbba-4497-8153-db9f3dbe9de9","Type":"ContainerStarted","Data":"0cce0e9f7d29cac12ba11d0983c21799f46ca41bf3d8c84f7bb9ca71daa4a4fb"} Jul 23 09:22:26.666633 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.666594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" event={"ID":"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2","Type":"ContainerStarted","Data":"cf696c413c1a648ac472d7359c607a910bcef8a7d0a31382d0b73e97002d970c"} Jul 23 09:22:26.668424 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.668394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" event={"ID":"b00f2c3c-20bb-483a-9d51-624bd6f4990f","Type":"ContainerStarted","Data":"7c7fdaa3884e2b72655b1c767743611c9f8176f39df9dcfbd8e12255c16c5a94"} Jul 23 09:22:26.668651 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.668625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:26.670314 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.670276 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qzpsq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 09:22:26.670435 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.670352 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" podUID="b00f2c3c-20bb-483a-9d51-624bd6f4990f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 09:22:26.725178 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:26.725126 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" podStartSLOduration=2.727851947 podStartE2EDuration="5.725107734s" totalImagesPullingTime="2.997255787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:26.722124052 +0000 UTC m=+326.932307204" watchObservedRunningTime="2026-07-23 09:22:26.725107734 +0000 UTC m=+326.935290883" Jul 23 09:22:27.675145 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:27.675114 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qzpsq" Jul 23 09:22:28.682523 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:28.682485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" event={"ID":"936103be-6c95-4ee3-a6ee-0abd6aa2ebe2","Type":"ContainerStarted","Data":"1539c8eae14020444d29c5c8272dd369c0ee57460296ce8d6d600a42fef55269"} Jul 23 09:22:28.702844 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:28.702754 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" podStartSLOduration=2.201367223 podStartE2EDuration="4.702736381s" totalImagesPullingTime="2.501369158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:28.700849263 +0000 UTC m=+328.911032415" watchObservedRunningTime="2026-07-23 09:22:28.702736381 +0000 UTC m=+328.912919534" Jul 23 09:22:29.629896 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:29.629861 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pr4jf" Jul 23 09:22:29.689228 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:29.689192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" event={"ID":"b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b","Type":"ContainerStarted","Data":"1bf9d9c1712938ba5b5418205cd454f4478e77180c25e4a45341e00dffa63da3"} Jul 23 09:22:29.689725 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:29.689391 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:22:29.689725 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:29.689630 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:22:29.726229 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:29.726111 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" podStartSLOduration=2.202864296 podStartE2EDuration="5.726093567s" totalImagesPullingTime="3.523229271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:29.724553189 +0000 UTC m=+329.934736342" watchObservedRunningTime="2026-07-23 09:22:29.726093567 +0000 UTC m=+329.936276719" Jul 23 09:22:30.599576 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:30.599541 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:22:30.599642 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:30.599614 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:22:30.599689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:30.599644 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:22:30.695078 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:30.695014 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" event={"ID":"9c15706b-bbba-4497-8153-db9f3dbe9de9","Type":"ContainerStarted","Data":"8314875f1b2f6e7242416a851cd5b3473ca729de6f1dad6326018f1c4db2c381"} Jul 23 09:22:30.719926 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:30.719865 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" podStartSLOduration=0.775034814 podStartE2EDuration="4.719842543s" totalImagesPullingTime="3.944807729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:22:30.717417094 +0000 UTC m=+330.927600244" watchObservedRunningTime="2026-07-23 09:22:30.719842543 +0000 UTC m=+330.930025694" Jul 23 09:22:31.491439 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:31.491396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:31.496141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:31.496121 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:31.699611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:31.699580 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:22:31.700527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:22:31.700507 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bt5dh" Jul 23 09:23:00.697637 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:00.697603 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hbdvn" Jul 23 09:23:00.700823 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:00.700803 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-kzdd9" Jul 23 09:23:58.403429 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.403392 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp"] Jul 23 09:23:58.406443 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.406422 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:23:58.408821 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.408801 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 09:23:58.408969 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.408946 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 09:23:58.409649 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.409629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bsgz7\"" Jul 23 09:23:58.416205 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.416185 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp"] Jul 23 09:23:58.523665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.523631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlskr\" (UniqueName: \"kubernetes.io/projected/47ef682f-6fb0-41e4-8344-a0acd870e08c-kube-api-access-vlskr\") pod \"authorino-operator-75cc8d5cb7-nwnmp\" (UID: \"47ef682f-6fb0-41e4-8344-a0acd870e08c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:23:58.624769 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.624720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlskr\" (UniqueName: \"kubernetes.io/projected/47ef682f-6fb0-41e4-8344-a0acd870e08c-kube-api-access-vlskr\") pod \"authorino-operator-75cc8d5cb7-nwnmp\" (UID: \"47ef682f-6fb0-41e4-8344-a0acd870e08c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:23:58.632914 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.632892 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlskr\" (UniqueName: \"kubernetes.io/projected/47ef682f-6fb0-41e4-8344-a0acd870e08c-kube-api-access-vlskr\") pod \"authorino-operator-75cc8d5cb7-nwnmp\" (UID: \"47ef682f-6fb0-41e4-8344-a0acd870e08c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:23:58.717587 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.717546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:23:58.849987 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:58.849952 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp"] Jul 23 09:23:58.853007 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:23:58.852978 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ef682f_6fb0_41e4_8344_a0acd870e08c.slice/crio-24793fe258e2b7b543c7d747d3211a7b726ffa76e0a151bd923758b3db43f9e4 WatchSource:0}: Error finding container 24793fe258e2b7b543c7d747d3211a7b726ffa76e0a151bd923758b3db43f9e4: Status 404 returned error can't find the container with id 24793fe258e2b7b543c7d747d3211a7b726ffa76e0a151bd923758b3db43f9e4 Jul 23 09:23:59.051396 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:23:59.051299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" event={"ID":"47ef682f-6fb0-41e4-8344-a0acd870e08c","Type":"ContainerStarted","Data":"24793fe258e2b7b543c7d747d3211a7b726ffa76e0a151bd923758b3db43f9e4"} Jul 23 09:24:02.069495 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:02.069459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" event={"ID":"47ef682f-6fb0-41e4-8344-a0acd870e08c","Type":"ContainerStarted","Data":"e72f165f12f848568dd01dc9b73c31b7aee0197e7eef1967b54e9d3dd4420933"} Jul 23 09:24:02.069894 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:02.069608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:24:13.076497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:13.076465 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" Jul 23 09:24:13.100776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:13.100724 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nwnmp" podStartSLOduration=12.66100509 podStartE2EDuration="15.100706272s" totalImagesPullingTime="2.439701182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:24:02.097449029 +0000 UTC m=+422.307632179" watchObservedRunningTime="2026-07-23 09:24:13.100706272 +0000 UTC m=+433.310889421" Jul 23 09:24:53.255166 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.255137 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44"] Jul 23 09:24:53.258246 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.258225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.260665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.260641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4q7sx\"" Jul 23 09:24:53.276099 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.276076 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44"] Jul 23 09:24:53.384244 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384416 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384416 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384416 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384416 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97crf\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-kube-api-access-97crf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.384704 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.384607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485512 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97crf\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-kube-api-access-97crf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485700 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485700 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.485816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486015 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.485993 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486084 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486084 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486193 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486345 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486299 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486463 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486463 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.486563 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.486545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.488805 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.488781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.489939 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.489914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.496597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.496573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.498742 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.498718 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97crf\" (UniqueName: \"kubernetes.io/projected/f0af4e63-1e3a-4400-8677-4f2e3bac89ac-kube-api-access-97crf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kvl44\" (UID: \"f0af4e63-1e3a-4400-8677-4f2e3bac89ac\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.568711 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.568635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:53.709114 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.709089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44"] Jul 23 09:24:53.711199 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:24:53.711175 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0af4e63_1e3a_4400_8677_4f2e3bac89ac.slice/crio-49fcf8777adcc33fa4c8377588193daf730627ffca2bd14c2ef5424c96e647d7 WatchSource:0}: Error finding container 49fcf8777adcc33fa4c8377588193daf730627ffca2bd14c2ef5424c96e647d7: Status 404 returned error can't find the container with id 49fcf8777adcc33fa4c8377588193daf730627ffca2bd14c2ef5424c96e647d7 Jul 23 09:24:53.713193 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.713163 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:24:53.713256 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.713242 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:24:53.713295 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:53.713280 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:24:54.290508 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:54.290464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" event={"ID":"f0af4e63-1e3a-4400-8677-4f2e3bac89ac","Type":"ContainerStarted","Data":"0e312f313f94ce5e84dd5f7e5becb7891d670f1eb86fa2cdc802c4d435083b15"} Jul 23 09:24:54.290876 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:54.290532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" event={"ID":"f0af4e63-1e3a-4400-8677-4f2e3bac89ac","Type":"ContainerStarted","Data":"49fcf8777adcc33fa4c8377588193daf730627ffca2bd14c2ef5424c96e647d7"} Jul 23 09:24:54.311836 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:54.311774 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" podStartSLOduration=1.3117572800000001 podStartE2EDuration="1.31175728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:24:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:24:54.309895012 +0000 UTC m=+474.520078174" watchObservedRunningTime="2026-07-23 09:24:54.31175728 +0000 UTC m=+474.521940430" Jul 23 09:24:54.569010 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:54.568918 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:54.573550 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:54.573525 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:55.295424 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:55.295396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:24:55.296246 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:24:55.296229 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:25:46.772060 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.772019 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:25:46.775503 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.775480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:46.780468 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.780447 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 09:25:46.780468 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.780464 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 09:25:46.780638 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.780483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nprx9\"" Jul 23 09:25:46.786884 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.786861 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:25:46.843401 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.843361 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcdrg\" (UniqueName: \"kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:46.843401 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.843402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:46.944112 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.944074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcdrg\" (UniqueName: \"kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:46.944112 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.944110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:46.944363 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:25:46.944216 2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 23 09:25:46.944363 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:25:46.944269 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert podName:d8fc3144-c0cd-4d65-84c8-c405127fa2f0 nodeName:}" failed. No retries permitted until 2026-07-23 09:25:47.444252441 +0000 UTC m=+527.654435569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert") pod "maas-controller-68c9f8fb88-4d475" (UID: "d8fc3144-c0cd-4d65-84c8-c405127fa2f0") : secret "maas-controller-webhook-cert" not found Jul 23 09:25:46.956905 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:46.956881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcdrg\" (UniqueName: \"kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:47.447915 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:47.447878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:47.450504 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:47.450479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") pod \"maas-controller-68c9f8fb88-4d475\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:47.686776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:47.686738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:47.821572 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:47.821550 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:25:48.508626 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:48.508562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c9f8fb88-4d475" event={"ID":"d8fc3144-c0cd-4d65-84c8-c405127fa2f0","Type":"ContainerStarted","Data":"1c9890f6a6fcbc363fae29d88fafb526b550bbbfa7b79336c8fb46830423e1a8"} Jul 23 09:25:50.519213 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:50.519181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c9f8fb88-4d475" event={"ID":"d8fc3144-c0cd-4d65-84c8-c405127fa2f0","Type":"ContainerStarted","Data":"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451"} Jul 23 09:25:50.519661 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:50.519298 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:25:50.536912 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:25:50.536864 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68c9f8fb88-4d475" podStartSLOduration=2.342227186 podStartE2EDuration="4.536850052s" totalImagesPullingTime="2.194622866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:25:50.536111069 +0000 UTC m=+530.746294212" watchObservedRunningTime="2026-07-23 09:25:50.536850052 +0000 UTC m=+530.747033254" Jul 23 09:26:01.526407 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:01.526379 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:26:22.583777 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.583728 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn"] Jul 23 09:26:22.587022 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.587005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.589630 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.589603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zh7zc\"" Jul 23 09:26:22.589756 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.589649 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 09:26:22.589756 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.589656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 09:26:22.589756 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.589658 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 23 09:26:22.599211 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.599186 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn"] Jul 23 09:26:22.736574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.736539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-openshift-service-ca\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.736574 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.736577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.736812 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.736601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9p7k\" (UniqueName: \"kubernetes.io/projected/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-kube-api-access-f9p7k\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.838092 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.838007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-openshift-service-ca\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.838092 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.838045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.838092 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.838065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9p7k\" (UniqueName: \"kubernetes.io/projected/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-kube-api-access-f9p7k\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.838322 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:26:22.838177 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 23 09:26:22.838322 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:26:22.838270 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls podName:8aa15f19-8f7d-4acc-bb5b-480ef0cca55f nodeName:}" failed. No retries permitted until 2026-07-23 09:26:23.338245583 +0000 UTC m=+563.548428711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls") pod "maas-api-77fd7f6cc4-j9cmn" (UID: "8aa15f19-8f7d-4acc-bb5b-480ef0cca55f") : secret "maas-api-serving-cert" not found Jul 23 09:26:22.838795 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.838775 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-openshift-service-ca\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:22.852642 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:22.852603 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9p7k\" (UniqueName: \"kubernetes.io/projected/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-kube-api-access-f9p7k\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:23.344672 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:23.344624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:23.347116 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:23.347093 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa15f19-8f7d-4acc-bb5b-480ef0cca55f-maas-api-tls\") pod \"maas-api-77fd7f6cc4-j9cmn\" (UID: \"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f\") " pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:23.498065 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:23.498034 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:23.636467 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:23.636413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn"] Jul 23 09:26:23.638691 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:26:23.638655 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa15f19_8f7d_4acc_bb5b_480ef0cca55f.slice/crio-8ad4869be1a3607b845f4a6c3b949d093736e76cf8b3d2cdc72c887913a6c326 WatchSource:0}: Error finding container 8ad4869be1a3607b845f4a6c3b949d093736e76cf8b3d2cdc72c887913a6c326: Status 404 returned error can't find the container with id 8ad4869be1a3607b845f4a6c3b949d093736e76cf8b3d2cdc72c887913a6c326 Jul 23 09:26:23.655394 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:23.655362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" event={"ID":"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f","Type":"ContainerStarted","Data":"8ad4869be1a3607b845f4a6c3b949d093736e76cf8b3d2cdc72c887913a6c326"} Jul 23 09:26:25.665472 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:25.665437 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" event={"ID":"8aa15f19-8f7d-4acc-bb5b-480ef0cca55f","Type":"ContainerStarted","Data":"9af557cca071831ae8be5944517a0d888c1b75b73e527b56f32aa7ef7b742463"} Jul 23 09:26:25.665882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:25.665564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:25.684492 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:25.684430 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" podStartSLOduration=2.033889718 podStartE2EDuration="3.684411335s" totalImagesPullingTime="1.650521617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:26:25.682730673 +0000 UTC m=+565.892913823" watchObservedRunningTime="2026-07-23 09:26:25.684411335 +0000 UTC m=+565.894594485" Jul 23 09:26:30.689681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.689648 2562 generic.go:356] "Generic (PLEG): container finished" podID="f0af4e63-1e3a-4400-8677-4f2e3bac89ac" containerID="0e312f313f94ce5e84dd5f7e5becb7891d670f1eb86fa2cdc802c4d435083b15" exitCode=0 Jul 23 09:26:30.690067 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.689719 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" event={"ID":"f0af4e63-1e3a-4400-8677-4f2e3bac89ac","Type":"ContainerDied","Data":"0e312f313f94ce5e84dd5f7e5becb7891d670f1eb86fa2cdc802c4d435083b15"} Jul 23 09:26:30.690107 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.690087 2562 scope.go:117] "RemoveContainer" containerID="0e312f313f94ce5e84dd5f7e5becb7891d670f1eb86fa2cdc802c4d435083b15" Jul 23 09:26:30.691132 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.691101 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:26:30.691207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.691189 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:26:30.691257 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:30.691240 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:26:31.676268 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:31.676237 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-77fd7f6cc4-j9cmn" Jul 23 09:26:31.696008 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:31.695975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" event={"ID":"f0af4e63-1e3a-4400-8677-4f2e3bac89ac","Type":"ContainerStarted","Data":"f3dd46fdb845d6984b8176f34a559737750054447a91229660b1bf6e2f6de4e3"} Jul 23 09:26:32.569598 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:32.569554 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:26:32.569856 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:32.569835 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kvl44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body= Jul 23 09:26:32.569921 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:32.569893 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" podUID="f0af4e63-1e3a-4400-8677-4f2e3bac89ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" Jul 23 09:26:33.569387 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:33.569318 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kvl44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body= Jul 23 09:26:33.569767 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:33.569421 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" podUID="f0af4e63-1e3a-4400-8677-4f2e3bac89ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" Jul 23 09:26:34.569279 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:34.569240 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kvl44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body= Jul 23 09:26:34.569510 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:34.569305 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" podUID="f0af4e63-1e3a-4400-8677-4f2e3bac89ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" Jul 23 09:26:35.617544 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:35.617501 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:26:35.617988 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:35.617914 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:26:35.618958 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:35.618932 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kvl44" Jul 23 09:26:50.795586 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.795539 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq"] Jul 23 09:26:50.801630 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.801608 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.804198 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.804177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 09:26:50.805059 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.805032 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 09:26:50.805059 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.805053 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 09:26:50.805200 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.805119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hw9h8\"" Jul 23 09:26:50.809654 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.809630 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq"] Jul 23 09:26:50.896476 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.896651 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.896651 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.896651 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssn9\" (UniqueName: \"kubernetes.io/projected/36b260b3-6136-4c8f-8823-a99c4d100b56-kube-api-access-nssn9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.896765 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b260b3-6136-4c8f-8823-a99c4d100b56-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.896765 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.896750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997496 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997496 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssn9\" (UniqueName: \"kubernetes.io/projected/36b260b3-6136-4c8f-8823-a99c4d100b56-kube-api-access-nssn9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b260b3-6136-4c8f-8823-a99c4d100b56-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997961 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997937 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.997961 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997951 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:50.998065 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:50.997998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:51.000058 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.000025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b260b3-6136-4c8f-8823-a99c4d100b56-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:51.000242 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.000222 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b260b3-6136-4c8f-8823-a99c4d100b56-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:51.007509 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.007490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssn9\" (UniqueName: \"kubernetes.io/projected/36b260b3-6136-4c8f-8823-a99c4d100b56-kube-api-access-nssn9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq\" (UID: \"36b260b3-6136-4c8f-8823-a99c4d100b56\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:51.112847 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.112755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:26:51.249998 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.249973 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq"] Jul 23 09:26:51.251933 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:26:51.251906 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b260b3_6136_4c8f_8823_a99c4d100b56.slice/crio-b40ed82ebea2348539bbf7a2dfc48dbee1cbf89f7175dca2a85899c8cb192c9a WatchSource:0}: Error finding container b40ed82ebea2348539bbf7a2dfc48dbee1cbf89f7175dca2a85899c8cb192c9a: Status 404 returned error can't find the container with id b40ed82ebea2348539bbf7a2dfc48dbee1cbf89f7175dca2a85899c8cb192c9a Jul 23 09:26:51.783323 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:51.783289 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" event={"ID":"36b260b3-6136-4c8f-8823-a99c4d100b56","Type":"ContainerStarted","Data":"b40ed82ebea2348539bbf7a2dfc48dbee1cbf89f7175dca2a85899c8cb192c9a"} Jul 23 09:26:52.791814 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:52.791778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" event={"ID":"36b260b3-6136-4c8f-8823-a99c4d100b56","Type":"ContainerStarted","Data":"04000f4dcb0312b8249afa47bad3e2a4bbce9059b5179edc3ab957af9f1dc543"} Jul 23 09:26:52.813738 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:26:52.813688 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" podStartSLOduration=1.666738929 podStartE2EDuration="2.813673224s" totalImagesPullingTime="1.146934295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:26:52.812878036 +0000 UTC m=+593.023061187" watchObservedRunningTime="2026-07-23 09:26:52.813673224 +0000 UTC m=+593.023856374" Jul 23 09:27:00.199149 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:00.199114 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:27:00.200372 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:00.200323 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:27:01.113250 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:01.113209 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:27:01.113461 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:01.113290 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:27:01.127905 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:01.127880 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:27:01.846954 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:01.846924 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq" Jul 23 09:27:07.872130 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.872092 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7"] Jul 23 09:27:07.876362 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.876342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.881029 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.881011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 23 09:27:07.887285 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.887265 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7"] Jul 23 09:27:07.945528 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rz9p\" (UniqueName: \"kubernetes.io/projected/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-kube-api-access-4rz9p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.945717 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945562 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.945717 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.945717 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.945875 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:07.945875 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:07.945790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.046980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.046942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.046980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.046980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047227 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047227 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047227 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rz9p\" (UniqueName: \"kubernetes.io/projected/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-kube-api-access-4rz9p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047227 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047481 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047541 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.047575 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.047546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.049440 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.049421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.049758 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.049738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.057100 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.057078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rz9p\" (UniqueName: \"kubernetes.io/projected/bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674-kube-api-access-4rz9p\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7\" (UID: \"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.187611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.187577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:08.318853 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.318817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7"] Jul 23 09:27:08.321480 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:27:08.321453 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdab7b4e_1c8d_41c1_9dc9_455c4ce6f674.slice/crio-117301c85d98fac478b7a77d1212984ac71ccfbacc64ac4618c7f18b98914e4f WatchSource:0}: Error finding container 117301c85d98fac478b7a77d1212984ac71ccfbacc64ac4618c7f18b98914e4f: Status 404 returned error can't find the container with id 117301c85d98fac478b7a77d1212984ac71ccfbacc64ac4618c7f18b98914e4f Jul 23 09:27:08.871850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.871769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" event={"ID":"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674","Type":"ContainerStarted","Data":"7dc593cc7a88913038f4303a5139273c7672b045b30a12d7a145acd1fe725026"} Jul 23 09:27:08.871850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.871848 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" event={"ID":"bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674","Type":"ContainerStarted","Data":"117301c85d98fac478b7a77d1212984ac71ccfbacc64ac4618c7f18b98914e4f"} Jul 23 09:27:08.910553 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:08.910501 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" podStartSLOduration=1.6917744890000002 podStartE2EDuration="1.910486094s" totalImagesPullingTime="218.711605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:27:08.909065279 +0000 UTC m=+609.119248429" watchObservedRunningTime="2026-07-23 09:27:08.910486094 +0000 UTC m=+609.120669243" Jul 23 09:27:18.187979 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:18.187941 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:18.188482 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:18.188024 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:18.200285 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:18.200263 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:18.925647 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:18.925621 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7" Jul 23 09:27:19.079142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.079098 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4"] Jul 23 09:27:19.085235 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.085205 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.087596 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.087577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 09:27:19.092977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.092954 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4"] Jul 23 09:27:19.249215 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.249689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163133b3-8c33-445e-8ed8-8e002986c4fe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.249689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.249689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.249689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249405 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mlkj\" (UniqueName: \"kubernetes.io/projected/163133b3-8c33-445e-8ed8-8e002986c4fe-kube-api-access-7mlkj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.249689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.249448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350563 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350563 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350803 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mlkj\" (UniqueName: \"kubernetes.io/projected/163133b3-8c33-445e-8ed8-8e002986c4fe-kube-api-access-7mlkj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350803 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350803 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.350803 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.350680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163133b3-8c33-445e-8ed8-8e002986c4fe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.351046 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.351023 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.351104 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.351034 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.351165 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.351143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.353136 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.353100 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/163133b3-8c33-445e-8ed8-8e002986c4fe-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.353381 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.353363 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163133b3-8c33-445e-8ed8-8e002986c4fe-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.359304 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.359277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mlkj\" (UniqueName: \"kubernetes.io/projected/163133b3-8c33-445e-8ed8-8e002986c4fe-kube-api-access-7mlkj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qxq4\" (UID: \"163133b3-8c33-445e-8ed8-8e002986c4fe\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.396154 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.396120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:19.535075 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.535043 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4"] Jul 23 09:27:19.536894 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:27:19.536868 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163133b3_8c33_445e_8ed8_8e002986c4fe.slice/crio-3fe4c5bf4049d7e4ee85ca394f92f2869f9a5032a5a06ac657920efd771bfd5a WatchSource:0}: Error finding container 3fe4c5bf4049d7e4ee85ca394f92f2869f9a5032a5a06ac657920efd771bfd5a: Status 404 returned error can't find the container with id 3fe4c5bf4049d7e4ee85ca394f92f2869f9a5032a5a06ac657920efd771bfd5a Jul 23 09:27:19.538405 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.538385 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 09:27:19.920788 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.920749 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" event={"ID":"163133b3-8c33-445e-8ed8-8e002986c4fe","Type":"ContainerStarted","Data":"5e3ca471d4d978948916fdab62154c2fbd6b823dfe51ec3e244f7077aa6071d8"} Jul 23 09:27:19.920959 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.920818 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" event={"ID":"163133b3-8c33-445e-8ed8-8e002986c4fe","Type":"ContainerStarted","Data":"3fe4c5bf4049d7e4ee85ca394f92f2869f9a5032a5a06ac657920efd771bfd5a"} Jul 23 09:27:19.940964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:19.940916 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" podStartSLOduration=0.729257813 podStartE2EDuration="940.898522ms" totalImagesPullingTime="211.640709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:27:19.939647859 +0000 UTC m=+620.149831013" watchObservedRunningTime="2026-07-23 09:27:19.940898522 +0000 UTC m=+620.151081672" Jul 23 09:27:29.396779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:29.396688 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:29.396779 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:29.396774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:29.409414 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:29.409379 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:27:29.976816 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:27:29.976786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qxq4" Jul 23 09:29:23.490860 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.490821 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:29:23.491427 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.491093 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68c9f8fb88-4d475" podUID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" containerName="manager" containerID="cri-o://1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451" gracePeriod=30 Jul 23 09:29:23.743619 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.743558 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:29:23.805996 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.805964 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcdrg\" (UniqueName: \"kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg\") pod \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " Jul 23 09:29:23.806166 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.806075 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") pod \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\" (UID: \"d8fc3144-c0cd-4d65-84c8-c405127fa2f0\") " Jul 23 09:29:23.808255 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.808226 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg" (OuterVolumeSpecName: "kube-api-access-fcdrg") pod "d8fc3144-c0cd-4d65-84c8-c405127fa2f0" (UID: "d8fc3144-c0cd-4d65-84c8-c405127fa2f0"). InnerVolumeSpecName "kube-api-access-fcdrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:29:23.808414 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.808295 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d8fc3144-c0cd-4d65-84c8-c405127fa2f0" (UID: "d8fc3144-c0cd-4d65-84c8-c405127fa2f0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:29:23.907449 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.907416 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcdrg\" (UniqueName: \"kubernetes.io/projected/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-kube-api-access-fcdrg\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:29:23.907449 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:23.907444 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d8fc3144-c0cd-4d65-84c8-c405127fa2f0-webhook-cert\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:29:24.455384 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.455320 2562 generic.go:356] "Generic (PLEG): container finished" podID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" containerID="1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451" exitCode=0 Jul 23 09:29:24.455384 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.455379 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c9f8fb88-4d475" Jul 23 09:29:24.455596 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.455405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c9f8fb88-4d475" event={"ID":"d8fc3144-c0cd-4d65-84c8-c405127fa2f0","Type":"ContainerDied","Data":"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451"} Jul 23 09:29:24.455596 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.455456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c9f8fb88-4d475" event={"ID":"d8fc3144-c0cd-4d65-84c8-c405127fa2f0","Type":"ContainerDied","Data":"1c9890f6a6fcbc363fae29d88fafb526b550bbbfa7b79336c8fb46830423e1a8"} Jul 23 09:29:24.455596 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.455492 2562 scope.go:117] "RemoveContainer" containerID="1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451" Jul 23 09:29:24.464238 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.464224 2562 scope.go:117] "RemoveContainer" containerID="1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451" Jul 23 09:29:24.464575 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:29:24.464555 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451\": container with ID starting with 1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451 not found: ID does not exist" containerID="1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451" Jul 23 09:29:24.464629 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.464583 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451"} err="failed to get container status \"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451\": rpc error: code = NotFound desc = could not find container \"1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451\": container with ID starting with 1cbc99efd6108bf048fe7c664c4b0fe5d5bc403fbb86e3eabb685820339bf451 not found: ID does not exist" Jul 23 09:29:24.474398 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.474376 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:29:24.477850 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:24.477830 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68c9f8fb88-4d475"] Jul 23 09:29:26.345115 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:29:26.345081 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" path="/var/lib/kubelet/pods/d8fc3144-c0cd-4d65-84c8-c405127fa2f0/volumes" Jul 23 09:32:00.225665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:32:00.225633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:32:00.225665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:32:00.225661 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:37:00.251654 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:37:00.251614 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:37:00.253689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:37:00.253667 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:42:00.274163 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:42:00.274131 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:42:00.277751 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:42:00.277700 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:44:37.329993 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.329958 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:44:37.330611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.330315 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" containerName="manager" Jul 23 09:44:37.330611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.330357 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" containerName="manager" Jul 23 09:44:37.330611 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.330427 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fc3144-c0cd-4d65-84c8-c405127fa2f0" containerName="manager" Jul 23 09:44:37.333256 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.333238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.335969 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.335947 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0c364a76-openshift-default-dockercfg-9mgb5\"" Jul 23 09:44:37.358261 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.358233 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:44:37.428795 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.428756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxfvx\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.428997 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.428851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.428997 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.428910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.428997 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.428973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.429169 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.429086 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.429623 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.429599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.429721 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.429681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.429782 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.429739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.429837 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.429807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530718 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530718 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530832 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.530962 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.530861 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531196 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531148 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531196 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531294 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxfvx\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531294 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531294 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531245 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.531707 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.531663 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.533496 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.533474 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.533797 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.533775 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.540662 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.540635 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.540820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.540804 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxfvx\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx\") pod \"e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.646253 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.646158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:37.792859 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.792823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:44:37.796341 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:44:37.796295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7b9100_5694_4eb6_ba2e_1bf254800c8f.slice/crio-967b97b694404dc6b149fad58fbef680f61c9c66f2f9a0bd8eada102c5b83027 WatchSource:0}: Error finding container 967b97b694404dc6b149fad58fbef680f61c9c66f2f9a0bd8eada102c5b83027: Status 404 returned error can't find the container with id 967b97b694404dc6b149fad58fbef680f61c9c66f2f9a0bd8eada102c5b83027 Jul 23 09:44:37.798885 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.798851 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:37.798986 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.798939 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:37.799047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:37.798985 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:38.408784 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:38.408748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerStarted","Data":"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca"} Jul 23 09:44:38.409194 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:38.408802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerStarted","Data":"967b97b694404dc6b149fad58fbef680f61c9c66f2f9a0bd8eada102c5b83027"} Jul 23 09:44:38.647012 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:38.646970 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:38.652021 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:38.651981 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:38.684997 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:38.684890 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podStartSLOduration=1.684874337 podStartE2EDuration="1.684874337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:44:38.570309375 +0000 UTC m=+1658.780492525" watchObservedRunningTime="2026-07-23 09:44:38.684874337 +0000 UTC m=+1658.895057486" Jul 23 09:44:39.413302 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:39.413270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:39.414194 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:39.414176 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:54.843917 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:54.843884 2562 patch_prober.go:28] interesting pod/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": read tcp 10.134.0.2:51100->10.134.0.28:15021: read: connection reset by peer" start-of-body= Jul 23 09:44:54.844287 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:54.843960 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": read tcp 10.134.0.2:51100->10.134.0.28:15021: read: connection reset by peer" Jul 23 09:44:55.486664 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.486628 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerID="4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca" exitCode=0 Jul 23 09:44:55.486852 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.486716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerDied","Data":"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca"} Jul 23 09:44:55.487184 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.487169 2562 scope.go:117] "RemoveContainer" containerID="4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca" Jul 23 09:44:55.488231 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.488200 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:55.488367 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.488285 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:55.488367 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:55.488356 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:44:56.493148 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:56.493106 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerStarted","Data":"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66"} Jul 23 09:44:56.646567 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:56.646526 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:44:56.646818 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:56.646791 2562 patch_prober.go:28] interesting pod/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 23 09:44:56.646913 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:56.646850 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 23 09:44:57.647616 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:57.647573 2562 patch_prober.go:28] interesting pod/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 23 09:44:57.648076 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:57.647630 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 23 09:44:58.647180 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:58.647140 2562 patch_prober.go:28] interesting pod/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 23 09:44:58.647378 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:58.647198 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 23 09:44:59.646909 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:59.646870 2562 patch_prober.go:28] interesting pod/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 23 09:44:59.647291 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:44:59.646926 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 23 09:45:00.650538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:00.650504 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:45:00.650990 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:00.650789 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:45:00.651610 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:00.651593 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:45:21.053317 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:21.053235 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:45:21.053879 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:21.053489 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" containerID="cri-o://3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66" gracePeriod=30 Jul 23 09:45:26.306244 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.306217 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:45:26.378695 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378552 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.378695 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378636 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.378695 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.378695 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378690 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378715 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxfvx\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378755 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378778 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378824 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378847 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy\") pod \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\" (UID: \"6f7b9100-5694-4eb6-ba2e-1bf254800c8f\") " Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378853 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:45:26.379142 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.378935 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:45:26.379493 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.379170 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-certs\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.379493 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.379191 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-workload-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.379493 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.379462 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:45:26.379805 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.379587 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:45:26.379805 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.379758 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:45:26.381441 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.381410 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx" (OuterVolumeSpecName: "kube-api-access-jxfvx") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "kube-api-access-jxfvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:45:26.381640 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.381480 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 09:45:26.381742 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.381718 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:45:26.381853 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.381760 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f7b9100-5694-4eb6-ba2e-1bf254800c8f" (UID: "6f7b9100-5694-4eb6-ba2e-1bf254800c8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:45:26.480721 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480680 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-credential-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480721 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480723 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istiod-ca-cert\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480721 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480737 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-token\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480965 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480748 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-envoy\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480965 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480760 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-podinfo\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480965 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480771 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-istio-data\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.480965 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.480785 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxfvx\" (UniqueName: \"kubernetes.io/projected/6f7b9100-5694-4eb6-ba2e-1bf254800c8f-kube-api-access-jxfvx\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:45:26.622720 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.622685 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerID="3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66" exitCode=0 Jul 23 09:45:26.622882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.622735 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" Jul 23 09:45:26.622882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.622767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerDied","Data":"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66"} Jul 23 09:45:26.622882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.622826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb" event={"ID":"6f7b9100-5694-4eb6-ba2e-1bf254800c8f","Type":"ContainerDied","Data":"967b97b694404dc6b149fad58fbef680f61c9c66f2f9a0bd8eada102c5b83027"} Jul 23 09:45:26.622882 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.622857 2562 scope.go:117] "RemoveContainer" containerID="3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66" Jul 23 09:45:26.632207 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.632183 2562 scope.go:117] "RemoveContainer" containerID="4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca" Jul 23 09:45:26.640368 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.640345 2562 scope.go:117] "RemoveContainer" containerID="3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66" Jul 23 09:45:26.640721 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:45:26.640701 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66\": container with ID starting with 3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66 not found: ID does not exist" containerID="3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66" Jul 23 09:45:26.640761 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.640731 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66"} err="failed to get container status \"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66\": rpc error: code = NotFound desc = could not find container \"3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66\": container with ID starting with 3620c522eaf7e0a9cb09ce0757def1ef55a2bd754803b30248e860dc5f089c66 not found: ID does not exist" Jul 23 09:45:26.640761 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.640751 2562 scope.go:117] "RemoveContainer" containerID="4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca" Jul 23 09:45:26.641011 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:45:26.640991 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca\": container with ID starting with 4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca not found: ID does not exist" containerID="4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca" Jul 23 09:45:26.641047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.641018 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca"} err="failed to get container status \"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca\": rpc error: code = NotFound desc = could not find container \"4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca\": container with ID starting with 4ec4278693266abccc146d657b8672e08abe5d13efd7ff2db44e0e41175c70ca not found: ID does not exist" Jul 23 09:45:26.648614 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.648588 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:45:26.653595 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:26.653567 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0c364a76-openshift-default-cdfd96959-p8psb"] Jul 23 09:45:28.344425 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:45:28.344395 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" path="/var/lib/kubelet/pods/6f7b9100-5694-4eb6-ba2e-1bf254800c8f/volumes" Jul 23 09:46:12.732981 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.732947 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:12.739386 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.739349 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:12.739386 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.739385 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:12.739576 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.739419 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:12.739576 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.739428 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:12.739783 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.739607 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:12.744160 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.744139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.747235 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.747213 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2ee6db4d-serving-cert\"" Jul 23 09:46:12.748238 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.748211 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:12.888751 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.888711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nd97\" (UniqueName: \"kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.888937 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.888773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.888937 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.888863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.989533 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.989451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.989533 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.989521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nd97\" (UniqueName: \"kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.989722 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.989550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:12.989722 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:46:12.989665 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-serving-cert: secret "maas-api-e2e-ait-2ee6db4d-serving-cert" not found Jul 23 09:46:12.989722 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:46:12.989720 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls podName:33baadc5-c94f-42d6-8422-a78c2ab2e8f2 nodeName:}" failed. No retries permitted until 2026-07-23 09:46:13.489704133 +0000 UTC m=+1753.699887261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls") pod "maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" (UID: "33baadc5-c94f-42d6-8422-a78c2ab2e8f2") : secret "maas-api-e2e-ait-2ee6db4d-serving-cert" not found Jul 23 09:46:12.990254 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:12.990231 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:13.001315 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.001277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nd97\" (UniqueName: \"kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:13.494057 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.494014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:13.496633 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.496604 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") pod \"maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:13.656501 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.656465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:13.810139 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.810094 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:13.811071 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:46:13.811039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33baadc5_c94f_42d6_8422_a78c2ab2e8f2.slice/crio-8ec94fadabf17412515071ec6f09dd6e28e329ccd8125e29abe1fb6674851bc6 WatchSource:0}: Error finding container 8ec94fadabf17412515071ec6f09dd6e28e329ccd8125e29abe1fb6674851bc6: Status 404 returned error can't find the container with id 8ec94fadabf17412515071ec6f09dd6e28e329ccd8125e29abe1fb6674851bc6 Jul 23 09:46:13.812533 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.812512 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 09:46:13.836763 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:13.836734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" event={"ID":"33baadc5-c94f-42d6-8422-a78c2ab2e8f2","Type":"ContainerStarted","Data":"8ec94fadabf17412515071ec6f09dd6e28e329ccd8125e29abe1fb6674851bc6"} Jul 23 09:46:14.843111 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:14.843072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" event={"ID":"33baadc5-c94f-42d6-8422-a78c2ab2e8f2","Type":"ContainerStarted","Data":"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94"} Jul 23 09:46:14.843552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:14.843197 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:14.861519 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:14.861459 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" podStartSLOduration=2.365456988 podStartE2EDuration="2.86144096s" totalImagesPullingTime="495.983972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:46:14.860611505 +0000 UTC m=+1755.070794651" watchObservedRunningTime="2026-07-23 09:46:14.86144096 +0000 UTC m=+1755.071624110" Jul 23 09:46:20.852829 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:20.852798 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:49.111597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.111561 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:49.112200 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.111817 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" podUID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" containerName="maas-api" containerID="cri-o://e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94" gracePeriod=30 Jul 23 09:46:49.351992 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.351967 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:49.401613 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.401543 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nd97\" (UniqueName: \"kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97\") pod \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " Jul 23 09:46:49.401613 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.401585 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") pod \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " Jul 23 09:46:49.401792 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.401629 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca\") pod \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\" (UID: \"33baadc5-c94f-42d6-8422-a78c2ab2e8f2\") " Jul 23 09:46:49.402005 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.401984 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33baadc5-c94f-42d6-8422-a78c2ab2e8f2" (UID: "33baadc5-c94f-42d6-8422-a78c2ab2e8f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:46:49.403961 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.403933 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97" (OuterVolumeSpecName: "kube-api-access-9nd97") pod "33baadc5-c94f-42d6-8422-a78c2ab2e8f2" (UID: "33baadc5-c94f-42d6-8422-a78c2ab2e8f2"). InnerVolumeSpecName "kube-api-access-9nd97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:46:49.404059 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.403956 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33baadc5-c94f-42d6-8422-a78c2ab2e8f2" (UID: "33baadc5-c94f-42d6-8422-a78c2ab2e8f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:46:49.502904 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.502867 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nd97\" (UniqueName: \"kubernetes.io/projected/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-kube-api-access-9nd97\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:46:49.502904 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.502896 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-maas-api-tls\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:46:49.502904 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.502911 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33baadc5-c94f-42d6-8422-a78c2ab2e8f2-openshift-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:46:49.998250 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.998215 2562 generic.go:356] "Generic (PLEG): container finished" podID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" containerID="e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94" exitCode=0 Jul 23 09:46:49.998467 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.998275 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" Jul 23 09:46:49.998467 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.998300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" event={"ID":"33baadc5-c94f-42d6-8422-a78c2ab2e8f2","Type":"ContainerDied","Data":"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94"} Jul 23 09:46:49.998467 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.998400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9" event={"ID":"33baadc5-c94f-42d6-8422-a78c2ab2e8f2","Type":"ContainerDied","Data":"8ec94fadabf17412515071ec6f09dd6e28e329ccd8125e29abe1fb6674851bc6"} Jul 23 09:46:49.998467 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:49.998424 2562 scope.go:117] "RemoveContainer" containerID="e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94" Jul 23 09:46:50.007297 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:50.007280 2562 scope.go:117] "RemoveContainer" containerID="e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94" Jul 23 09:46:50.007572 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:46:50.007552 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94\": container with ID starting with e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94 not found: ID does not exist" containerID="e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94" Jul 23 09:46:50.007650 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:50.007583 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94"} err="failed to get container status \"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94\": rpc error: code = NotFound desc = could not find container \"e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94\": container with ID starting with e1eebb25824a88ce6f7c002c38400eda6b1e6d8005c1ed8b855281246cbb7f94 not found: ID does not exist" Jul 23 09:46:50.022618 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:50.022589 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:50.028853 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:50.028833 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2ee6db4d-788b44cd7c-s6lw9"] Jul 23 09:46:50.345426 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:50.345302 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" path="/var/lib/kubelet/pods/33baadc5-c94f-42d6-8422-a78c2ab2e8f2/volumes" Jul 23 09:46:57.972623 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.972529 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:46:57.973102 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.973078 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" containerName="maas-api" Jul 23 09:46:57.973102 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.973098 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" containerName="maas-api" Jul 23 09:46:57.973211 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.973192 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="33baadc5-c94f-42d6-8422-a78c2ab2e8f2" containerName="maas-api" Jul 23 09:46:57.973211 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.973209 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7b9100-5694-4eb6-ba2e-1bf254800c8f" containerName="istio-proxy" Jul 23 09:46:57.976491 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.976468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:57.979012 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:57.978986 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0b80ea18-serving-cert\"" Jul 23 09:46:58.000279 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.000251 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:46:58.079585 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.079550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.079585 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.079586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.079795 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.079669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj72n\" (UniqueName: \"kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.180894 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.180855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj72n\" (UniqueName: \"kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.181079 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.180907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.181177 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.181154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.181584 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.181552 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.183661 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.183642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.189511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.189476 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj72n\" (UniqueName: \"kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n\") pod \"maas-api-e2e-derive-0b80ea18-86f668d675-65qnh\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.288358 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.288252 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:58.429203 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:58.429072 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:46:58.431525 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:46:58.431496 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda78016_f3d5_466a_be90_352aae79b32b.slice/crio-9d96f29bbc1fd1f75a84cad83f587155450556ae36e918eb8b2fc1004a4d65e1 WatchSource:0}: Error finding container 9d96f29bbc1fd1f75a84cad83f587155450556ae36e918eb8b2fc1004a4d65e1: Status 404 returned error can't find the container with id 9d96f29bbc1fd1f75a84cad83f587155450556ae36e918eb8b2fc1004a4d65e1 Jul 23 09:46:59.041815 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:59.041783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" event={"ID":"bda78016-f3d5-466a-be90-352aae79b32b","Type":"ContainerStarted","Data":"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f"} Jul 23 09:46:59.042161 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:59.041836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" event={"ID":"bda78016-f3d5-466a-be90-352aae79b32b","Type":"ContainerStarted","Data":"9d96f29bbc1fd1f75a84cad83f587155450556ae36e918eb8b2fc1004a4d65e1"} Jul 23 09:46:59.042161 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:59.041891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:46:59.061156 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:46:59.061056 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" podStartSLOduration=1.6846409740000001 podStartE2EDuration="2.061039484s" totalImagesPullingTime="376.39851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:46:59.059660392 +0000 UTC m=+1799.269843543" watchObservedRunningTime="2026-07-23 09:46:59.061039484 +0000 UTC m=+1799.271222633" Jul 23 09:47:00.298771 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:00.298741 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:47:00.302646 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:00.302626 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:47:05.053156 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:05.053116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:47:33.303456 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.303420 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:47:33.303960 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.303641 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" podUID="bda78016-f3d5-466a-be90-352aae79b32b" containerName="maas-api" containerID="cri-o://43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f" gracePeriod=30 Jul 23 09:47:33.560382 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.560289 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:47:33.593967 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.593935 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls\") pod \"bda78016-f3d5-466a-be90-352aae79b32b\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " Jul 23 09:47:33.594178 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.594003 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj72n\" (UniqueName: \"kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n\") pod \"bda78016-f3d5-466a-be90-352aae79b32b\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " Jul 23 09:47:33.594178 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.594023 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca\") pod \"bda78016-f3d5-466a-be90-352aae79b32b\" (UID: \"bda78016-f3d5-466a-be90-352aae79b32b\") " Jul 23 09:47:33.594445 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.594412 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bda78016-f3d5-466a-be90-352aae79b32b" (UID: "bda78016-f3d5-466a-be90-352aae79b32b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:47:33.596274 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.596241 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n" (OuterVolumeSpecName: "kube-api-access-mj72n") pod "bda78016-f3d5-466a-be90-352aae79b32b" (UID: "bda78016-f3d5-466a-be90-352aae79b32b"). InnerVolumeSpecName "kube-api-access-mj72n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:47:33.596274 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.596251 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bda78016-f3d5-466a-be90-352aae79b32b" (UID: "bda78016-f3d5-466a-be90-352aae79b32b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 09:47:33.695392 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.695308 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bda78016-f3d5-466a-be90-352aae79b32b-maas-api-tls\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:47:33.695392 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.695383 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj72n\" (UniqueName: \"kubernetes.io/projected/bda78016-f3d5-466a-be90-352aae79b32b-kube-api-access-mj72n\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:47:33.695392 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:33.695410 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda78016-f3d5-466a-be90-352aae79b32b-openshift-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:47:34.191768 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.191729 2562 generic.go:356] "Generic (PLEG): container finished" podID="bda78016-f3d5-466a-be90-352aae79b32b" containerID="43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f" exitCode=0 Jul 23 09:47:34.191768 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.191769 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" Jul 23 09:47:34.192047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.191824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" event={"ID":"bda78016-f3d5-466a-be90-352aae79b32b","Type":"ContainerDied","Data":"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f"} Jul 23 09:47:34.192047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.191883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh" event={"ID":"bda78016-f3d5-466a-be90-352aae79b32b","Type":"ContainerDied","Data":"9d96f29bbc1fd1f75a84cad83f587155450556ae36e918eb8b2fc1004a4d65e1"} Jul 23 09:47:34.192047 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.191914 2562 scope.go:117] "RemoveContainer" containerID="43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f" Jul 23 09:47:34.201068 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.201048 2562 scope.go:117] "RemoveContainer" containerID="43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f" Jul 23 09:47:34.201381 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:47:34.201360 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f\": container with ID starting with 43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f not found: ID does not exist" containerID="43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f" Jul 23 09:47:34.201432 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.201390 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f"} err="failed to get container status \"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f\": rpc error: code = NotFound desc = could not find container \"43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f\": container with ID starting with 43c74011889de3c5a0a2777e5cf928140b1a0f2125d469c9ab60cb6ebff86f3f not found: ID does not exist" Jul 23 09:47:34.213341 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.213311 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:47:34.219429 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.219404 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0b80ea18-86f668d675-65qnh"] Jul 23 09:47:34.345252 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:47:34.345219 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda78016-f3d5-466a-be90-352aae79b32b" path="/var/lib/kubelet/pods/bda78016-f3d5-466a-be90-352aae79b32b/volumes" Jul 23 09:49:30.151186 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.151141 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556"] Jul 23 09:49:30.151641 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.151523 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda78016-f3d5-466a-be90-352aae79b32b" containerName="maas-api" Jul 23 09:49:30.151641 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.151534 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda78016-f3d5-466a-be90-352aae79b32b" containerName="maas-api" Jul 23 09:49:30.151641 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.151591 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda78016-f3d5-466a-be90-352aae79b32b" containerName="maas-api" Jul 23 09:49:30.154597 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.154574 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:30.157023 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.157002 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 09:49:30.161567 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.161529 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556"] Jul 23 09:49:30.228258 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.228222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18ba59c9-wh556\" (UID: \"28d1486c-65c1-4e6f-a5c0-bc45be60d516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:30.329254 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.329212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18ba59c9-wh556\" (UID: \"28d1486c-65c1-4e6f-a5c0-bc45be60d516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:30.329897 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.329872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18ba59c9-wh556\" (UID: \"28d1486c-65c1-4e6f-a5c0-bc45be60d516\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:30.467527 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.467491 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:30.609612 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.609578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556"] Jul 23 09:49:30.612448 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:49:30.612414 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d1486c_65c1_4e6f_a5c0_bc45be60d516.slice/crio-2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45 WatchSource:0}: Error finding container 2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45: Status 404 returned error can't find the container with id 2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45 Jul 23 09:49:30.698980 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:30.698851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" event={"ID":"28d1486c-65c1-4e6f-a5c0-bc45be60d516","Type":"ContainerStarted","Data":"2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45"} Jul 23 09:49:33.714723 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:33.714686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" event={"ID":"28d1486c-65c1-4e6f-a5c0-bc45be60d516","Type":"ContainerStarted","Data":"ceef0141d487c901f16a33accbd540b0d48818b61d179a937cb6a0ce1801290d"} Jul 23 09:49:33.732013 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:33.731940 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" podStartSLOduration=1.235411581 podStartE2EDuration="3.731913901s" totalImagesPullingTime="2.49650232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:49:33.730209393 +0000 UTC m=+1953.940392557" watchObservedRunningTime="2026-07-23 09:49:33.731913901 +0000 UTC m=+1953.942097051" Jul 23 09:49:34.720569 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:34.720530 2562 generic.go:356] "Generic (PLEG): container finished" podID="28d1486c-65c1-4e6f-a5c0-bc45be60d516" containerID="ceef0141d487c901f16a33accbd540b0d48818b61d179a937cb6a0ce1801290d" exitCode=0 Jul 23 09:49:34.720990 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:34.720592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" event={"ID":"28d1486c-65c1-4e6f-a5c0-bc45be60d516","Type":"ContainerDied","Data":"ceef0141d487c901f16a33accbd540b0d48818b61d179a937cb6a0ce1801290d"} Jul 23 09:49:35.861206 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:35.861175 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:35.983072 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:35.982969 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca\") pod \"28d1486c-65c1-4e6f-a5c0-bc45be60d516\" (UID: \"28d1486c-65c1-4e6f-a5c0-bc45be60d516\") " Jul 23 09:49:35.983395 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:35.983371 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28d1486c-65c1-4e6f-a5c0-bc45be60d516" (UID: "28d1486c-65c1-4e6f-a5c0-bc45be60d516"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:49:36.084415 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:36.084382 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28d1486c-65c1-4e6f-a5c0-bc45be60d516-maas-api-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:49:36.730146 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:36.730109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" event={"ID":"28d1486c-65c1-4e6f-a5c0-bc45be60d516","Type":"ContainerDied","Data":"2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45"} Jul 23 09:49:36.730146 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:36.730139 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556" Jul 23 09:49:36.730392 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:36.730165 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bc703a20fbc0db75b1d79b30a249427bcc5db8ec5677835ad74e5115d16bd45" Jul 23 09:49:45.221949 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:45.221900 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556"] Jul 23 09:49:45.223403 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:45.223383 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18ba59c9-wh556"] Jul 23 09:49:46.344623 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:49:46.344593 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d1486c-65c1-4e6f-a5c0-bc45be60d516" path="/var/lib/kubelet/pods/28d1486c-65c1-4e6f-a5c0-bc45be60d516/volumes" Jul 23 09:50:58.657036 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.656998 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt"] Jul 23 09:50:58.657523 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.657371 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d1486c-65c1-4e6f-a5c0-bc45be60d516" containerName="revoke-keys" Jul 23 09:50:58.657523 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.657383 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d1486c-65c1-4e6f-a5c0-bc45be60d516" containerName="revoke-keys" Jul 23 09:50:58.657523 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.657438 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d1486c-65c1-4e6f-a5c0-bc45be60d516" containerName="revoke-keys" Jul 23 09:50:58.660373 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.660354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:50:58.662687 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.662666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 09:50:58.667600 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.667575 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt"] Jul 23 09:50:58.723628 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.723591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt\" (UID: \"ec455593-bcfa-4112-8ba2-3bbb0cd58885\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:50:58.824295 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.824259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt\" (UID: \"ec455593-bcfa-4112-8ba2-3bbb0cd58885\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:50:58.824911 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.824893 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt\" (UID: \"ec455593-bcfa-4112-8ba2-3bbb0cd58885\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:50:58.971470 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:58.971399 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:50:59.107658 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:50:59.107633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt"] Jul 23 09:50:59.110106 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:50:59.110073 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec455593_bcfa_4112_8ba2_3bbb0cd58885.slice/crio-170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f WatchSource:0}: Error finding container 170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f: Status 404 returned error can't find the container with id 170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f Jul 23 09:51:00.101393 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:00.101295 2562 generic.go:356] "Generic (PLEG): container finished" podID="ec455593-bcfa-4112-8ba2-3bbb0cd58885" containerID="0144fa316521df1a20a07da3020aef9be37ee76a2056f42ca0764f579e58a79b" exitCode=0 Jul 23 09:51:00.101393 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:00.101370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" event={"ID":"ec455593-bcfa-4112-8ba2-3bbb0cd58885","Type":"ContainerDied","Data":"0144fa316521df1a20a07da3020aef9be37ee76a2056f42ca0764f579e58a79b"} Jul 23 09:51:00.101782 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:00.101451 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" event={"ID":"ec455593-bcfa-4112-8ba2-3bbb0cd58885","Type":"ContainerStarted","Data":"170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f"} Jul 23 09:51:01.241653 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:01.241628 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:51:01.348911 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:01.348869 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca\") pod \"ec455593-bcfa-4112-8ba2-3bbb0cd58885\" (UID: \"ec455593-bcfa-4112-8ba2-3bbb0cd58885\") " Jul 23 09:51:01.349234 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:01.349209 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ec455593-bcfa-4112-8ba2-3bbb0cd58885" (UID: "ec455593-bcfa-4112-8ba2-3bbb0cd58885"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:51:01.450300 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:01.450268 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec455593-bcfa-4112-8ba2-3bbb0cd58885-maas-api-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:51:02.111509 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:02.111471 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" Jul 23 09:51:02.111687 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:02.111468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt" event={"ID":"ec455593-bcfa-4112-8ba2-3bbb0cd58885","Type":"ContainerDied","Data":"170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f"} Jul 23 09:51:02.111687 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:02.111602 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="170b07f5e77d7b9cf6480cda2da5fd1cd8bdf08710e5bbb64986b0d7bddb0a6f" Jul 23 09:51:18.732395 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:18.732356 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt"] Jul 23 09:51:18.735245 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:18.735221 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18c2a482-wl6qt"] Jul 23 09:51:20.344964 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:51:20.344891 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec455593-bcfa-4112-8ba2-3bbb0cd58885" path="/var/lib/kubelet/pods/ec455593-bcfa-4112-8ba2-3bbb0cd58885/volumes" Jul 23 09:52:00.322473 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:52:00.322442 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:52:00.327849 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:52:00.327826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:53:11.470380 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.470323 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:53:11.471023 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.470704 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec455593-bcfa-4112-8ba2-3bbb0cd58885" containerName="revoke-keys" Jul 23 09:53:11.471023 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.470723 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec455593-bcfa-4112-8ba2-3bbb0cd58885" containerName="revoke-keys" Jul 23 09:53:11.471023 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.470848 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec455593-bcfa-4112-8ba2-3bbb0cd58885" containerName="revoke-keys" Jul 23 09:53:11.473992 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.473967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.476896 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.476871 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-64f44e-openshift-default-dockercfg-85dbk\"" Jul 23 09:53:11.486250 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.486226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:53:11.591951 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.591877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.591964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592141 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6n8q\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592421 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592240 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592421 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.592421 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.592316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693274 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693274 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693538 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6n8q\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693881 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693719 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693881 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693810 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.693881 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.693856 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.694090 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.694066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.694258 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.694241 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.695941 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.695924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.696074 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.696058 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.702557 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.702533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.702842 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.702821 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6n8q\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q\") pod \"e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.785108 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.785002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:11.933282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.933108 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:53:11.936008 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:53:11.935981 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1109d4fb_6262_4a8e_863c_cd7ed3f575dd.slice/crio-8bb0efe14fd71651f75306da73952a3eee4eb60d0edb36e607f2621f46c0ab79 WatchSource:0}: Error finding container 8bb0efe14fd71651f75306da73952a3eee4eb60d0edb36e607f2621f46c0ab79: Status 404 returned error can't find the container with id 8bb0efe14fd71651f75306da73952a3eee4eb60d0edb36e607f2621f46c0ab79 Jul 23 09:53:11.938187 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.938154 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:11.938272 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.938233 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:11.938350 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:11.938276 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:12.689262 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:12.689222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerStarted","Data":"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924"} Jul 23 09:53:12.689658 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:12.689280 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerStarted","Data":"8bb0efe14fd71651f75306da73952a3eee4eb60d0edb36e607f2621f46c0ab79"} Jul 23 09:53:12.711553 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:12.711501 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podStartSLOduration=1.711484666 podStartE2EDuration="1.711484666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:53:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:53:12.709985602 +0000 UTC m=+2172.920168815" watchObservedRunningTime="2026-07-23 09:53:12.711484666 +0000 UTC m=+2172.921667815" Jul 23 09:53:12.786193 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:12.786151 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:12.791198 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:12.791170 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:13.693739 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:13.693710 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:13.694808 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:13.694787 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:26.763854 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.763819 2562 generic.go:356] "Generic (PLEG): container finished" podID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerID="a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924" exitCode=0 Jul 23 09:53:26.764275 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.763873 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerDied","Data":"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924"} Jul 23 09:53:26.764275 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.764233 2562 scope.go:117] "RemoveContainer" containerID="a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924" Jul 23 09:53:26.765200 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.765172 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:26.765254 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.765241 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:26.765291 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:26.765272 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:53:27.770272 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:27.770240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerStarted","Data":"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46"} Jul 23 09:53:27.785154 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:27.785123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:27.786132 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:27.786111 2562 patch_prober.go:28] interesting pod/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 23 09:53:27.786229 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:27.786168 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 23 09:53:28.694091 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:28.694054 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:28.785539 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:28.785505 2562 patch_prober.go:28] interesting pod/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 23 09:53:28.785984 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:28.785562 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 23 09:53:29.785567 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:29.785530 2562 patch_prober.go:28] interesting pod/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 23 09:53:29.785968 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:29.785628 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 23 09:53:30.786017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:30.785981 2562 patch_prober.go:28] interesting pod/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 23 09:53:30.786425 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:30.786045 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 23 09:53:31.801854 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:31.801822 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:53:31.803037 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:53:31.803011 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:54:35.135073 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.135033 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd"] Jul 23 09:54:35.138442 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.138416 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:35.141158 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.141135 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 09:54:35.146946 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.146926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd"] Jul 23 09:54:35.244884 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.244846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd\" (UID: \"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:35.345681 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.345638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd\" (UID: \"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:35.346241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.346222 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd\" (UID: \"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:35.449549 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.449510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:35.596922 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:35.596892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd"] Jul 23 09:54:35.600184 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:54:35.600154 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b90d35f_3481_4b6b_b9aa_a9d8304ba50e.slice/crio-b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38 WatchSource:0}: Error finding container b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38: Status 404 returned error can't find the container with id b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38 Jul 23 09:54:36.086552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:36.086466 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" event={"ID":"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e","Type":"ContainerStarted","Data":"2c990f66b4661f1473abc0b606aedd41a66b5eca940f9c971b84aa17789721f2"} Jul 23 09:54:36.086552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:36.086530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" event={"ID":"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e","Type":"ContainerStarted","Data":"b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38"} Jul 23 09:54:36.111739 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:36.111693 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" podStartSLOduration=1.111676579 podStartE2EDuration="1.111676579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:54:36.109188473 +0000 UTC m=+2256.319371623" watchObservedRunningTime="2026-07-23 09:54:36.111676579 +0000 UTC m=+2256.321859729" Jul 23 09:54:37.092731 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:37.092691 2562 generic.go:356] "Generic (PLEG): container finished" podID="0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" containerID="2c990f66b4661f1473abc0b606aedd41a66b5eca940f9c971b84aa17789721f2" exitCode=0 Jul 23 09:54:37.093106 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:37.092771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" event={"ID":"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e","Type":"ContainerDied","Data":"2c990f66b4661f1473abc0b606aedd41a66b5eca940f9c971b84aa17789721f2"} Jul 23 09:54:38.236304 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:38.236277 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:38.273881 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:38.273848 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca\") pod \"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e\" (UID: \"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e\") " Jul 23 09:54:38.274177 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:38.274152 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" (UID: "0b90d35f-3481-4b6b-b9aa-a9d8304ba50e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:54:38.374567 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:38.374537 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e-maas-api-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:54:39.102790 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:39.102753 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" Jul 23 09:54:39.102790 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:39.102767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd" event={"ID":"0b90d35f-3481-4b6b-b9aa-a9d8304ba50e","Type":"ContainerDied","Data":"b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38"} Jul 23 09:54:39.103017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:39.102814 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b45a83e1ceadbe7c92a47f244e83f32f11855e44bd38af09b55472c4f5ccdd38" Jul 23 09:54:50.288547 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:50.288515 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd"] Jul 23 09:54:50.298496 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:50.298464 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16017a-8xkjd"] Jul 23 09:54:50.345427 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:54:50.345392 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" path="/var/lib/kubelet/pods/0b90d35f-3481-4b6b-b9aa-a9d8304ba50e/volumes" Jul 23 09:55:20.815973 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:20.815934 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:55:20.816521 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:20.816240 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" containerID="cri-o://ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46" gracePeriod=30 Jul 23 09:55:26.075827 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.075760 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:55:26.109295 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109353 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109442 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109489 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109773 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109516 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109773 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109572 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6n8q\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109773 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109612 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs\") pod \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\" (UID: \"1109d4fb-6262-4a8e-863c-cd7ed3f575dd\") " Jul 23 09:55:26.109905 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109770 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:55:26.109905 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109869 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-credential-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.110007 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109981 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:55:26.110143 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.109904 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:55:26.110585 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.110551 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:55:26.110585 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.110565 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:55:26.111757 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.111730 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 09:55:26.111843 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.111770 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:55:26.111843 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.111777 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:55:26.111929 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.111914 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q" (OuterVolumeSpecName: "kube-api-access-g6n8q") pod "1109d4fb-6262-4a8e-863c-cd7ed3f575dd" (UID: "1109d4fb-6262-4a8e-863c-cd7ed3f575dd"). InnerVolumeSpecName "kube-api-access-g6n8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:55:26.211017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.210972 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-envoy\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211010 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-podinfo\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211019 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-token\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211017 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211028 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istiod-ca-cert\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211036 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-istio-data\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211044 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211053 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6n8q\" (UniqueName: \"kubernetes.io/projected/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-kube-api-access-g6n8q\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.211282 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.211064 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1109d4fb-6262-4a8e-863c-cd7ed3f575dd-workload-certs\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:26.324160 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.324122 2562 generic.go:356] "Generic (PLEG): container finished" podID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerID="ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46" exitCode=0 Jul 23 09:55:26.324376 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.324178 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" Jul 23 09:55:26.324376 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.324212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerDied","Data":"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46"} Jul 23 09:55:26.324376 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.324321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf" event={"ID":"1109d4fb-6262-4a8e-863c-cd7ed3f575dd","Type":"ContainerDied","Data":"8bb0efe14fd71651f75306da73952a3eee4eb60d0edb36e607f2621f46c0ab79"} Jul 23 09:55:26.324551 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.324377 2562 scope.go:117] "RemoveContainer" containerID="ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46" Jul 23 09:55:26.333843 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.333820 2562 scope.go:117] "RemoveContainer" containerID="a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924" Jul 23 09:55:26.342768 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.342748 2562 scope.go:117] "RemoveContainer" containerID="ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46" Jul 23 09:55:26.343064 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:55:26.343030 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46\": container with ID starting with ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46 not found: ID does not exist" containerID="ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46" Jul 23 09:55:26.343138 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.343085 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46"} err="failed to get container status \"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46\": rpc error: code = NotFound desc = could not find container \"ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46\": container with ID starting with ac31c29acd7d033571ccf7334d6f57cf42f0ad491df1041bf7160e6f33ca6c46 not found: ID does not exist" Jul 23 09:55:26.343138 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.343120 2562 scope.go:117] "RemoveContainer" containerID="a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924" Jul 23 09:55:26.343450 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:55:26.343431 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924\": container with ID starting with a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924 not found: ID does not exist" containerID="a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924" Jul 23 09:55:26.343511 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.343456 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924"} err="failed to get container status \"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924\": rpc error: code = NotFound desc = could not find container \"a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924\": container with ID starting with a988e88ed2ae7d6c39085ddaf20ddc8cc6ea79f2dfed62b1eb657fbb51361924 not found: ID does not exist" Jul 23 09:55:26.405244 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.405211 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:55:26.421031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:26.420998 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-64f44e-openshift-default-747f6d75c-bcnqf"] Jul 23 09:55:28.344674 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:28.344634 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" path="/var/lib/kubelet/pods/1109d4fb-6262-4a8e-863c-cd7ed3f575dd/volumes" Jul 23 09:55:30.730563 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.730519 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:55:30.731035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.730994 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.731035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731014 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.731035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731026 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" containerName="revoke-keys" Jul 23 09:55:30.731035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731034 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" containerName="revoke-keys" Jul 23 09:55:30.731776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731069 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.731776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731079 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.731776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731156 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.731776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731166 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b90d35f-3481-4b6b-b9aa-a9d8304ba50e" containerName="revoke-keys" Jul 23 09:55:30.731776 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.731178 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1109d4fb-6262-4a8e-863c-cd7ed3f575dd" containerName="istio-proxy" Jul 23 09:55:30.735763 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.735739 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.738689 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.738668 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-10b365-openshift-default-dockercfg-77xj5\"" Jul 23 09:55:30.782939 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.782903 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:55:30.853957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.853922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.853957 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.853969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854140 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854241 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.854508 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.854243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vbk2\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955505 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955643 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.955716 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.955719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vbk2\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.956111 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.956067 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.956250 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.956220 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.956360 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.956286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.956428 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.956398 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.956481 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.956421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.958260 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.958232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.958476 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.958458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.972718 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.972681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vbk2\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:30.972844 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:30.972783 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token\") pod \"e2e-api-b-10b365-openshift-default-5684664d99-255kw\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:31.045835 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.045733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:31.196116 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.196078 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:55:31.199056 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:55:31.199032 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2635a88_7a06_4f70_8c7b_090cc44526fe.slice/crio-3034d40a619a254b827e7b00225f4a1d614f49063d82a4a7f50fc82f1d5e07b4 WatchSource:0}: Error finding container 3034d40a619a254b827e7b00225f4a1d614f49063d82a4a7f50fc82f1d5e07b4: Status 404 returned error can't find the container with id 3034d40a619a254b827e7b00225f4a1d614f49063d82a4a7f50fc82f1d5e07b4 Jul 23 09:55:31.201165 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.201130 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:31.201237 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.201195 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:31.201237 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.201222 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:31.351265 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.351164 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerStarted","Data":"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f"} Jul 23 09:55:31.351265 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.351228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerStarted","Data":"3034d40a619a254b827e7b00225f4a1d614f49063d82a4a7f50fc82f1d5e07b4"} Jul 23 09:55:31.376164 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:31.376099 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podStartSLOduration=1.376077891 podStartE2EDuration="1.376077891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:31.374349762 +0000 UTC m=+2311.584532910" watchObservedRunningTime="2026-07-23 09:55:31.376077891 +0000 UTC m=+2311.586261041" Jul 23 09:55:32.046628 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:32.046589 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:32.051477 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:32.051452 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:32.355838 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:32.355746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:32.356598 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:32.356577 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:46.163814 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.163771 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h"] Jul 23 09:55:46.167428 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.167406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:46.170919 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.170897 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 09:55:46.177243 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.177208 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h"] Jul 23 09:55:46.302583 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.302544 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h\" (UID: \"b3d12b58-c9e7-479c-9e6b-eae00de3c95b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:46.403826 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.403776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h\" (UID: \"b3d12b58-c9e7-479c-9e6b-eae00de3c95b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:46.404482 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.404463 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h\" (UID: \"b3d12b58-c9e7-479c-9e6b-eae00de3c95b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:46.479909 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.479809 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:46.679550 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:46.679514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h"] Jul 23 09:55:46.682780 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:55:46.682742 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d12b58_c9e7_479c_9e6b_eae00de3c95b.slice/crio-88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3 WatchSource:0}: Error finding container 88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3: Status 404 returned error can't find the container with id 88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3 Jul 23 09:55:47.423213 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:47.423164 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" event={"ID":"b3d12b58-c9e7-479c-9e6b-eae00de3c95b","Type":"ContainerStarted","Data":"a0b65f5ac085cb4506af1786e9a7f46371485cb58f536b84a4fb8332018af5bd"} Jul 23 09:55:47.423706 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:47.423280 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" event={"ID":"b3d12b58-c9e7-479c-9e6b-eae00de3c95b","Type":"ContainerStarted","Data":"88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3"} Jul 23 09:55:47.473501 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:47.473445 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" podStartSLOduration=1.473426668 podStartE2EDuration="1.473426668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:55:47.4702521 +0000 UTC m=+2327.680435249" watchObservedRunningTime="2026-07-23 09:55:47.473426668 +0000 UTC m=+2327.683609822" Jul 23 09:55:48.357265 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:48.357216 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 09:55:48.357445 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:48.357288 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 09:55:48.429789 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:48.429756 2562 generic.go:356] "Generic (PLEG): container finished" podID="b3d12b58-c9e7-479c-9e6b-eae00de3c95b" containerID="a0b65f5ac085cb4506af1786e9a7f46371485cb58f536b84a4fb8332018af5bd" exitCode=0 Jul 23 09:55:48.430199 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:48.429845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" event={"ID":"b3d12b58-c9e7-479c-9e6b-eae00de3c95b","Type":"ContainerDied","Data":"a0b65f5ac085cb4506af1786e9a7f46371485cb58f536b84a4fb8332018af5bd"} Jul 23 09:55:49.581935 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:49.581908 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:49.737425 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:49.737308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca\") pod \"b3d12b58-c9e7-479c-9e6b-eae00de3c95b\" (UID: \"b3d12b58-c9e7-479c-9e6b-eae00de3c95b\") " Jul 23 09:55:49.737655 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:49.737633 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b3d12b58-c9e7-479c-9e6b-eae00de3c95b" (UID: "b3d12b58-c9e7-479c-9e6b-eae00de3c95b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:55:49.838796 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:49.838760 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3d12b58-c9e7-479c-9e6b-eae00de3c95b-maas-api-service-ca\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:55:50.439993 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.439961 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" Jul 23 09:55:50.440210 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.439990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h" event={"ID":"b3d12b58-c9e7-479c-9e6b-eae00de3c95b","Type":"ContainerDied","Data":"88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3"} Jul 23 09:55:50.440210 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.440046 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88ca70420bcf2b022a0b4f1f8c4e6c0dec6bd0e395ade9d92184ebdaa8dc55c3" Jul 23 09:55:50.441680 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.441649 2562 generic.go:356] "Generic (PLEG): container finished" podID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerID="5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f" exitCode=0 Jul 23 09:55:50.441795 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.441711 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerDied","Data":"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f"} Jul 23 09:55:50.442116 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.442098 2562 scope.go:117] "RemoveContainer" containerID="5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f" Jul 23 09:55:50.443404 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.443365 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:50.443495 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.443469 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:50.443535 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:50.443503 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:55:51.447806 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:51.447773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerStarted","Data":"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3"} Jul 23 09:55:52.046373 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:52.046308 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:52.046593 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:52.046572 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 23 09:55:52.046641 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:52.046622 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 23 09:55:53.046760 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:53.046718 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 23 09:55:53.047161 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:53.046790 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 23 09:55:54.046090 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:54.046057 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 23 09:55:54.046259 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:54.046129 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 23 09:55:55.046722 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:55.046682 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 23 09:55:55.047122 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:55.046740 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 23 09:55:56.050277 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:56.050245 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:56.050697 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:56.050590 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:55:56.051296 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:55:56.051281 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:56:00.399237 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:00.399206 2562 scope.go:117] "RemoveContainer" containerID="ceef0141d487c901f16a33accbd540b0d48818b61d179a937cb6a0ce1801290d" Jul 23 09:56:06.271496 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:06.271461 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h"] Jul 23 09:56:06.274541 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:06.274514 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-62b18d-b2k6h"] Jul 23 09:56:06.344758 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:06.344721 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3d12b58-c9e7-479c-9e6b-eae00de3c95b" path="/var/lib/kubelet/pods/b3d12b58-c9e7-479c-9e6b-eae00de3c95b/volumes" Jul 23 09:56:39.938403 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:39.938361 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:56:39.939225 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:39.938647 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" containerID="cri-o://965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3" gracePeriod=30 Jul 23 09:56:41.052645 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:41.052605 2562 patch_prober.go:28] interesting pod/e2e-api-b-10b365-openshift-default-5684664d99-255kw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 09:56:41.053042 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:41.052662 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 09:56:45.201748 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.201690 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:56:45.352777 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352728 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.352777 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352778 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vbk2\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352817 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352843 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352930 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.352977 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353031 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353005 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353269 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353030 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:45.353269 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353041 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353269 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353115 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo\") pod \"c2635a88-7a06-4f70-8c7b-090cc44526fe\" (UID: \"c2635a88-7a06-4f70-8c7b-090cc44526fe\") " Jul 23 09:56:45.353480 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353290 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:45.353480 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353401 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:45.353480 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353443 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 09:56:45.353480 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353466 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-certs\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.353904 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.353487 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-credential-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.354210 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.354179 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:45.355253 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.355221 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2" (OuterVolumeSpecName: "kube-api-access-6vbk2") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "kube-api-access-6vbk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:56:45.355415 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.355375 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 09:56:45.355415 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.355385 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 09:56:45.355503 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.355430 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2635a88-7a06-4f70-8c7b-090cc44526fe" (UID: "c2635a88-7a06-4f70-8c7b-090cc44526fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454914 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-workload-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454946 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2635a88-7a06-4f70-8c7b-090cc44526fe-istiod-ca-cert\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454956 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-envoy\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454964 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-podinfo\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454972 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vbk2\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-kube-api-access-6vbk2\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454981 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-token\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.455011 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.454989 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2635a88-7a06-4f70-8c7b-090cc44526fe-istio-data\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 09:56:45.695507 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.695470 2562 generic.go:356] "Generic (PLEG): container finished" podID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerID="965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3" exitCode=0 Jul 23 09:56:45.695708 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.695527 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" Jul 23 09:56:45.695708 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.695605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerDied","Data":"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3"} Jul 23 09:56:45.695708 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.695650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw" event={"ID":"c2635a88-7a06-4f70-8c7b-090cc44526fe","Type":"ContainerDied","Data":"3034d40a619a254b827e7b00225f4a1d614f49063d82a4a7f50fc82f1d5e07b4"} Jul 23 09:56:45.695708 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.695678 2562 scope.go:117] "RemoveContainer" containerID="965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3" Jul 23 09:56:45.704977 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.704960 2562 scope.go:117] "RemoveContainer" containerID="5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f" Jul 23 09:56:45.712758 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.712739 2562 scope.go:117] "RemoveContainer" containerID="965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3" Jul 23 09:56:45.713018 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:56:45.712998 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3\": container with ID starting with 965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3 not found: ID does not exist" containerID="965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3" Jul 23 09:56:45.713052 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.713028 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3"} err="failed to get container status \"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3\": rpc error: code = NotFound desc = could not find container \"965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3\": container with ID starting with 965ccdbc45aaf17adf9cf721b34da1fa969be93fffd167541acab89bcad513f3 not found: ID does not exist" Jul 23 09:56:45.713052 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.713044 2562 scope.go:117] "RemoveContainer" containerID="5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f" Jul 23 09:56:45.713267 ip-10-0-132-192 kubenswrapper[2562]: E0723 09:56:45.713253 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f\": container with ID starting with 5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f not found: ID does not exist" containerID="5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f" Jul 23 09:56:45.713306 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.713273 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f"} err="failed to get container status \"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f\": rpc error: code = NotFound desc = could not find container \"5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f\": container with ID starting with 5dc60498dce31d6511bf43ca4527051d01f5e3ea7dbdaa4bc74321ca2bcc629f not found: ID does not exist" Jul 23 09:56:45.717940 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.717915 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:56:45.723304 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:45.723274 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-10b365-openshift-default-5684664d99-255kw"] Jul 23 09:56:46.344752 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:46.344722 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" path="/var/lib/kubelet/pods/c2635a88-7a06-4f70-8c7b-090cc44526fe/volumes" Jul 23 09:56:48.682116 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682083 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682418 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682429 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682439 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682445 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682458 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3d12b58-c9e7-479c-9e6b-eae00de3c95b" containerName="revoke-keys" Jul 23 09:56:48.682497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682463 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3d12b58-c9e7-479c-9e6b-eae00de3c95b" containerName="revoke-keys" Jul 23 09:56:48.682677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682513 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682523 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2635a88-7a06-4f70-8c7b-090cc44526fe" containerName="istio-proxy" Jul 23 09:56:48.682677 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.682530 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3d12b58-c9e7-479c-9e6b-eae00de3c95b" containerName="revoke-keys" Jul 23 09:56:48.686900 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.686872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.689630 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.689604 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0dc985-openshift-default-dockercfg-c5x84\"" Jul 23 09:56:48.705552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.705521 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 09:56:48.781182 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.781522 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.781652 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.781751 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr6d6\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.781848 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.781970 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.781952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.782077 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.782050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.782257 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.782093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.782257 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.782156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883635 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr6d6\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.883820 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884173 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.883828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884173 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.884088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884173 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.884110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884325 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.884243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884325 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.884291 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.884570 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.884523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.886239 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.886212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.886644 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.886624 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.892665 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.892639 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr6d6\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.894552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.894536 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token\") pod \"e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:48.997647 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:48.997550 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:49.148580 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.148552 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 09:56:49.150756 ip-10-0-132-192 kubenswrapper[2562]: W0723 09:56:49.150725 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe286da2_87bb_4ddd_9084_13f1dd685e0f.slice/crio-e086a88bcc9cc98c2e3efc8068a310f0434937f5db61b8de0298a0cec486b871 WatchSource:0}: Error finding container e086a88bcc9cc98c2e3efc8068a310f0434937f5db61b8de0298a0cec486b871: Status 404 returned error can't find the container with id e086a88bcc9cc98c2e3efc8068a310f0434937f5db61b8de0298a0cec486b871 Jul 23 09:56:49.153001 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.152970 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:56:49.153081 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.153045 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:56:49.153118 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.153080 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:56:49.716846 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.716805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerStarted","Data":"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac"} Jul 23 09:56:49.717269 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.716871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerStarted","Data":"e086a88bcc9cc98c2e3efc8068a310f0434937f5db61b8de0298a0cec486b871"} Jul 23 09:56:49.739901 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.739524 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podStartSLOduration=1.7395028940000001 podStartE2EDuration="1.739502894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 09:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 09:56:49.738724286 +0000 UTC m=+2389.948907438" watchObservedRunningTime="2026-07-23 09:56:49.739502894 +0000 UTC m=+2389.949686045" Jul 23 09:56:49.998557 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:49.998460 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:51.003384 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:51.003348 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:51.726103 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:51.726072 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:56:51.727146 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:56:51.727127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:57:00.355035 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:00.355005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:57:00.366934 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:00.366911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 09:57:00.438749 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:00.438704 2562 scope.go:117] "RemoveContainer" containerID="0144fa316521df1a20a07da3020aef9be37ee76a2056f42ca0764f579e58a79b" Jul 23 09:57:07.140525 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.139721 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:42648->10.134.0.37:15021: read: connection reset by peer" start-of-body= Jul 23 09:57:07.140525 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.139799 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:42648->10.134.0.37:15021: read: connection reset by peer" Jul 23 09:57:07.810497 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.810463 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerID="ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac" exitCode=0 Jul 23 09:57:07.810670 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.810532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerDied","Data":"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac"} Jul 23 09:57:07.810938 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.810924 2562 scope.go:117] "RemoveContainer" containerID="ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac" Jul 23 09:57:07.811902 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.811876 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:57:07.811992 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.811951 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:57:07.812029 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:07.811990 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 23 09:57:08.819129 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:08.819078 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerStarted","Data":"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497"} Jul 23 09:57:08.997764 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:08.997723 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:57:08.998003 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:08.997974 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 23 09:57:08.998093 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:08.998068 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 23 09:57:09.998791 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:09.998745 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 23 09:57:09.999275 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:09.998819 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 23 09:57:10.998156 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:10.998116 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 23 09:57:10.998446 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:10.998175 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 23 09:57:11.998160 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:11.998116 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 23 09:57:11.998599 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:11.998196 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 23 09:57:13.002092 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:13.002055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:57:13.002552 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:13.002361 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 09:57:13.003015 ip-10-0-132-192 kubenswrapper[2562]: I0723 09:57:13.003000 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 10:00:00.147413 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.147373 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5"] Jul 23 10:00:00.150778 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.150749 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" Jul 23 10:00:00.152309 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.152285 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm"] Jul 23 10:00:00.153988 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.153964 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 10:00:00.155903 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.155885 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx"] Jul 23 10:00:00.156038 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.156011 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" Jul 23 10:00:00.159065 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.159045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" Jul 23 10:00:00.165985 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.165953 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4hmw\"" Jul 23 10:00:00.166313 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.166286 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5"] Jul 23 10:00:00.171925 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.171883 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm"] Jul 23 10:00:00.172518 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.172497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" Jul 23 10:00:00.173004 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.172981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" Jul 23 10:00:00.174000 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.173977 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx"] Jul 23 10:00:00.185354 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.184957 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" Jul 23 10:00:00.364789 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.364754 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm"] Jul 23 10:00:00.368978 ip-10-0-132-192 kubenswrapper[2562]: W0723 10:00:00.368947 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4e0360_b7e1_455b_ba9a_85d58d139bdd.slice/crio-5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884 WatchSource:0}: Error finding container 5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884: Status 404 returned error can't find the container with id 5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884 Jul 23 10:00:00.591169 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.591139 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5"] Jul 23 10:00:00.592881 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.592858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx"] Jul 23 10:00:00.593206 ip-10-0-132-192 kubenswrapper[2562]: W0723 10:00:00.593186 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3eba13_336a_4d82_b544_8eff3e6a0a7d.slice/crio-a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6 WatchSource:0}: Error finding container a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6: Status 404 returned error can't find the container with id a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6 Jul 23 10:00:00.595106 ip-10-0-132-192 kubenswrapper[2562]: W0723 10:00:00.595079 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0c7eaf_1562_4240_b791_bd0f1c24a3f6.slice/crio-5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824 WatchSource:0}: Error finding container 5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824: Status 404 returned error can't find the container with id 5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824 Jul 23 10:00:00.612487 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.612385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" event={"ID":"cf0c7eaf-1562-4240-b791-bd0f1c24a3f6","Type":"ContainerStarted","Data":"5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824"} Jul 23 10:00:00.613674 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.613646 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" event={"ID":"3d3eba13-336a-4d82-b544-8eff3e6a0a7d","Type":"ContainerStarted","Data":"a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6"} Jul 23 10:00:00.615181 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:00.615120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" event={"ID":"be4e0360-b7e1-455b-ba9a-85d58d139bdd","Type":"ContainerStarted","Data":"5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884"} Jul 23 10:00:01.622316 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.622220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" event={"ID":"cf0c7eaf-1562-4240-b791-bd0f1c24a3f6","Type":"ContainerStarted","Data":"9fd17b1e6c4b5d525d00f9a475a34bb827bb2bddcd5d59eaaa5dd9f58992b95b"} Jul 23 10:00:01.623853 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.623828 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" event={"ID":"3d3eba13-336a-4d82-b544-8eff3e6a0a7d","Type":"ContainerStarted","Data":"034f4a2f11efb1830ecf4fc5e6d80da6748f9e4616952b25da322af35c990a8a"} Jul 23 10:00:01.625360 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.625317 2562 generic.go:356] "Generic (PLEG): container finished" podID="be4e0360-b7e1-455b-ba9a-85d58d139bdd" containerID="4d5e34e13530d30e9a3e6d7856404a104da89ad1dc7732ae86a051c9e301141b" exitCode=0 Jul 23 10:00:01.625486 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.625392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" event={"ID":"be4e0360-b7e1-455b-ba9a-85d58d139bdd","Type":"ContainerDied","Data":"4d5e34e13530d30e9a3e6d7856404a104da89ad1dc7732ae86a051c9e301141b"} Jul 23 10:00:01.639509 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.639454 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" podStartSLOduration=1.639439405 podStartE2EDuration="1.639439405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:00:01.63755018 +0000 UTC m=+2581.847733329" watchObservedRunningTime="2026-07-23 10:00:01.639439405 +0000 UTC m=+2581.849622555" Jul 23 10:00:01.667382 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:01.667306 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" podStartSLOduration=1.667289417 podStartE2EDuration="1.667289417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 10:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:00:01.666865373 +0000 UTC m=+2581.877048524" watchObservedRunningTime="2026-07-23 10:00:01.667289417 +0000 UTC m=+2581.877472567" Jul 23 10:00:02.631131 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:02.631097 2562 generic.go:356] "Generic (PLEG): container finished" podID="3d3eba13-336a-4d82-b544-8eff3e6a0a7d" containerID="034f4a2f11efb1830ecf4fc5e6d80da6748f9e4616952b25da322af35c990a8a" exitCode=0 Jul 23 10:00:02.631606 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:02.631201 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" event={"ID":"3d3eba13-336a-4d82-b544-8eff3e6a0a7d","Type":"ContainerDied","Data":"034f4a2f11efb1830ecf4fc5e6d80da6748f9e4616952b25da322af35c990a8a"} Jul 23 10:00:02.632750 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:02.632720 2562 generic.go:356] "Generic (PLEG): container finished" podID="cf0c7eaf-1562-4240-b791-bd0f1c24a3f6" containerID="9fd17b1e6c4b5d525d00f9a475a34bb827bb2bddcd5d59eaaa5dd9f58992b95b" exitCode=0 Jul 23 10:00:02.632854 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:02.632801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" event={"ID":"cf0c7eaf-1562-4240-b791-bd0f1c24a3f6","Type":"ContainerDied","Data":"9fd17b1e6c4b5d525d00f9a475a34bb827bb2bddcd5d59eaaa5dd9f58992b95b"} Jul 23 10:00:02.777946 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:02.777919 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" Jul 23 10:00:03.638693 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:03.638656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" event={"ID":"be4e0360-b7e1-455b-ba9a-85d58d139bdd","Type":"ContainerDied","Data":"5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884"} Jul 23 10:00:03.638693 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:03.638677 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746680-lglqm" Jul 23 10:00:03.639210 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:03.638709 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5278365b773a145d29fbf9791e9e3b39530f100f0d7b0311b0391bf3d83e5884" Jul 23 10:00:03.799622 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:03.799596 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" Jul 23 10:00:03.801625 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:03.801599 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" Jul 23 10:00:04.644688 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.644597 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" Jul 23 10:00:04.644688 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.644631 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx" event={"ID":"cf0c7eaf-1562-4240-b791-bd0f1c24a3f6","Type":"ContainerDied","Data":"5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824"} Jul 23 10:00:04.645171 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.644720 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ae8ed4997806e1c64e285ba0df757e1d806c46fee9136fcf241500b9d95c824" Jul 23 10:00:04.646038 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.646017 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" Jul 23 10:00:04.646165 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.646013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5" event={"ID":"3d3eba13-336a-4d82-b544-8eff3e6a0a7d","Type":"ContainerDied","Data":"a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6"} Jul 23 10:00:04.646165 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:00:04.646136 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a23a75b2ee3c05e90138122f6599116d5c7ac6399f06fdfce03c79d7eafb31d6" Jul 23 10:01:00.483103 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:01:00.483069 2562 scope.go:117] "RemoveContainer" containerID="2c990f66b4661f1473abc0b606aedd41a66b5eca940f9c971b84aa17789721f2" Jul 23 10:02:00.383067 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:02:00.383036 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 10:02:00.395960 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:02:00.395934 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4rwpq_2715e14e-0c79-4e62-99f6-305a33da96cb/ovn-acl-logging/0.log" Jul 23 10:02:00.501864 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:02:00.501831 2562 scope.go:117] "RemoveContainer" containerID="a0b65f5ac085cb4506af1786e9a7f46371485cb58f536b84a4fb8332018af5bd" Jul 23 10:03:06.419806 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:06.419765 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx"] Jul 23 10:03:06.427355 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:06.427310 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-efdc11-29746680-c52wx"] Jul 23 10:03:08.344594 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:08.344564 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf0c7eaf-1562-4240-b791-bd0f1c24a3f6" path="/var/lib/kubelet/pods/cf0c7eaf-1562-4240-b791-bd0f1c24a3f6/volumes" Jul 23 10:03:28.787048 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:28.786967 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5"] Jul 23 10:03:28.789754 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:28.789728 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0dc985-29746680-d7bb5"] Jul 23 10:03:30.344600 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:30.344566 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d3eba13-336a-4d82-b544-8eff3e6a0a7d" path="/var/lib/kubelet/pods/3d3eba13-336a-4d82-b544-8eff3e6a0a7d/volumes" Jul 23 10:03:39.938777 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:39.938737 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 10:03:39.939200 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:39.938968 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" containerID="cri-o://c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497" gracePeriod=30 Jul 23 10:03:43.005156 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:43.005117 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 10:03:43.005570 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:43.005191 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 10:03:45.197436 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.197409 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 10:03:45.311209 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311209 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311197 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311466 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311233 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311466 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311297 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311466 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311369 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311466 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311466 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311456 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 10:03:45.311726 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311476 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr6d6\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311726 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311501 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311726 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311543 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket\") pod \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\" (UID: \"fe286da2-87bb-4ddd-9084-13f1dd685e0f\") " Jul 23 10:03:45.311862 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311687 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:03:45.311862 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311735 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:03:45.311964 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311889 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:03:45.311964 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311908 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-credential-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.311964 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311929 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istiod-ca-cert\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.311964 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.311949 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-certs\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.312927 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.312891 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:03:45.313712 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.313679 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:03:45.313822 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.313720 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 10:03:45.313822 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.313726 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 10:03:45.314060 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.314039 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6" (OuterVolumeSpecName: "kube-api-access-qr6d6") pod "fe286da2-87bb-4ddd-9084-13f1dd685e0f" (UID: "fe286da2-87bb-4ddd-9084-13f1dd685e0f"). InnerVolumeSpecName "kube-api-access-qr6d6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 10:03:45.412930 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412885 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-workload-socket\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.412930 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412921 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-envoy\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.412930 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412930 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-token\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.412930 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412938 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-podinfo\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.413198 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412946 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr6d6\" (UniqueName: \"kubernetes.io/projected/fe286da2-87bb-4ddd-9084-13f1dd685e0f-kube-api-access-qr6d6\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.413198 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.412956 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe286da2-87bb-4ddd-9084-13f1dd685e0f-istio-data\") on node \"ip-10-0-132-192.ec2.internal\" DevicePath \"\"" Jul 23 10:03:45.681318 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.681282 2562 generic.go:356] "Generic (PLEG): container finished" podID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerID="c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497" exitCode=0 Jul 23 10:03:45.681495 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.681360 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" Jul 23 10:03:45.681495 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.681381 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerDied","Data":"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497"} Jul 23 10:03:45.681495 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.681440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz" event={"ID":"fe286da2-87bb-4ddd-9084-13f1dd685e0f","Type":"ContainerDied","Data":"e086a88bcc9cc98c2e3efc8068a310f0434937f5db61b8de0298a0cec486b871"} Jul 23 10:03:45.681495 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.681465 2562 scope.go:117] "RemoveContainer" containerID="c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497" Jul 23 10:03:45.693393 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.693362 2562 scope.go:117] "RemoveContainer" containerID="ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac" Jul 23 10:03:45.702563 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.702538 2562 scope.go:117] "RemoveContainer" containerID="c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497" Jul 23 10:03:45.702839 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.702819 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 10:03:45.702896 ip-10-0-132-192 kubenswrapper[2562]: E0723 10:03:45.702859 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497\": container with ID starting with c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497 not found: ID does not exist" containerID="c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497" Jul 23 10:03:45.702932 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.702887 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497"} err="failed to get container status \"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497\": rpc error: code = NotFound desc = could not find container \"c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497\": container with ID starting with c9ec72ac46963432da73dfbfc10f0f074c4b2d9053f6aa832e302ba931eba497 not found: ID does not exist" Jul 23 10:03:45.702932 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.702908 2562 scope.go:117] "RemoveContainer" containerID="ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac" Jul 23 10:03:45.703155 ip-10-0-132-192 kubenswrapper[2562]: E0723 10:03:45.703138 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac\": container with ID starting with ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac not found: ID does not exist" containerID="ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac" Jul 23 10:03:45.703208 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.703160 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac"} err="failed to get container status \"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac\": rpc error: code = NotFound desc = could not find container \"ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac\": container with ID starting with ed3d1249a9f3fe2f978fcb3bf158ac87c8400fdbbd281a2642d045d3a7be65ac not found: ID does not exist" Jul 23 10:03:45.707040 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:45.707010 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0dc985-openshift-default-78f6bb9679-djckz"] Jul 23 10:03:46.202989 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.202961 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzdd9_936103be-6c95-4ee3-a6ee-0abd6aa2ebe2/manager/0.log" Jul 23 10:03:46.321855 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.321827 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hbdvn_b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b/manager/0.log" Jul 23 10:03:46.346087 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.346057 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" path="/var/lib/kubelet/pods/fe286da2-87bb-4ddd-9084-13f1dd685e0f/volumes" Jul 23 10:03:46.702520 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.702495 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pr4jf_075a3e0d-dc49-43fa-ad2f-16aedc544bb8/manager/0.log" Jul 23 10:03:46.963385 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.963286 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d6699bc9-nb66c_718ba171-fc11-4ef7-98bc-774a0d45ebc5/copy-manifests/0.log" Jul 23 10:03:46.969628 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:46.969605 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d6699bc9-nb66c_718ba171-fc11-4ef7-98bc-774a0d45ebc5/manager/0.log" Jul 23 10:03:47.471882 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:47.471851 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-77fd7f6cc4-j9cmn_8aa15f19-8f7d-4acc-bb5b-480ef0cca55f/maas-api/0.log" Jul 23 10:03:47.820257 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:47.820183 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746680-lglqm_be4e0360-b7e1-455b-ba9a-85d58d139bdd/cleanup/0.log" Jul 23 10:03:49.454220 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:49.454187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nwnmp_47ef682f-6fb0-41e4-8344-a0acd870e08c/manager/0.log" Jul 23 10:03:50.552873 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:50.552832 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bt5dh_9c15706b-bbba-4497-8153-db9f3dbe9de9/istio-proxy/0.log" Jul 23 10:03:50.681190 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:50.681147 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qzpsq_b00f2c3c-20bb-483a-9d51-624bd6f4990f/discovery/0.log" Jul 23 10:03:50.914263 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:50.914226 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dccdf9c4-8bfm9_66a53353-2dc0-48ca-b973-82d6d050f1cb/kube-auth-proxy/0.log" Jul 23 10:03:51.051645 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:51.051620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/1.log" Jul 23 10:03:51.820593 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:51.820549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-zbszq_36b260b3-6136-4c8f-8823-a99c4d100b56/main/0.log" Jul 23 10:03:52.173823 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:52.173786 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fffkg7_bdab7b4e-1c8d-41c1-9dc9-455c4ce6f674/main/0.log" Jul 23 10:03:52.293318 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:03:52.293287 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5qxq4_163133b3-8c33-445e-8ed8-8e002986c4fe/main/0.log" Jul 23 10:04:01.051191 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:01.051152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jxtrx_1278b23a-b894-4fa2-8eef-497ee694573b/global-pull-secret-syncer/0.log" Jul 23 10:04:01.227116 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:01.227078 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rmwhw_3d1ba3cc-58aa-472c-8de6-b4ffb983f8ae/konnectivity-agent/0.log" Jul 23 10:04:01.277317 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:01.277287 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-192.ec2.internal_74882c1bf505da02ef19fd26790a16c7/haproxy/0.log" Jul 23 10:04:07.212595 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:07.212560 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nwnmp_47ef682f-6fb0-41e4-8344-a0acd870e08c/manager/0.log" Jul 23 10:04:09.835395 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:09.835364 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwzm_b3ddbb4e-b08a-49c2-ad06-62b3be3316c1/node-exporter/0.log" Jul 23 10:04:09.857627 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:09.857602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwzm_b3ddbb4e-b08a-49c2-ad06-62b3be3316c1/kube-rbac-proxy/0.log" Jul 23 10:04:09.882680 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:09.882650 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwzm_b3ddbb4e-b08a-49c2-ad06-62b3be3316c1/init-textfile/0.log" Jul 23 10:04:11.376291 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376255 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh"] Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376594 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d3eba13-336a-4d82-b544-8eff3e6a0a7d" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376606 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d3eba13-336a-4d82-b544-8eff3e6a0a7d" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376615 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4e0360-b7e1-455b-ba9a-85d58d139bdd" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376620 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4e0360-b7e1-455b-ba9a-85d58d139bdd" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376627 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0c7eaf-1562-4240-b791-bd0f1c24a3f6" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376633 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0c7eaf-1562-4240-b791-bd0f1c24a3f6" containerName="cleanup" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376663 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" Jul 23 10:04:11.376685 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376669 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" Jul 23 10:04:11.376921 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376725 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" Jul 23 10:04:11.376921 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376732 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d3eba13-336a-4d82-b544-8eff3e6a0a7d" containerName="cleanup" Jul 23 10:04:11.376921 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376740 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe286da2-87bb-4ddd-9084-13f1dd685e0f" containerName="istio-proxy" Jul 23 10:04:11.376921 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376747 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4e0360-b7e1-455b-ba9a-85d58d139bdd" containerName="cleanup" Jul 23 10:04:11.376921 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.376753 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0c7eaf-1562-4240-b791-bd0f1c24a3f6" containerName="cleanup" Jul 23 10:04:11.379708 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.379688 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.382277 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.382255 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qcmqp\"/\"kube-root-ca.crt\"" Jul 23 10:04:11.382392 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.382254 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qcmqp\"/\"openshift-service-ca.crt\"" Jul 23 10:04:11.383113 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.383099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qcmqp\"/\"default-dockercfg-7c6dg\"" Jul 23 10:04:11.388587 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.388565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh"] Jul 23 10:04:11.439817 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.439777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-podres\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.439993 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.439829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-proc\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.439993 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.439864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf9fn\" (UniqueName: \"kubernetes.io/projected/3a0418c3-cecc-47a3-8b60-96c14e793ada-kube-api-access-qf9fn\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.439993 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.439978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-sys\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.440101 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.440006 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-lib-modules\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540616 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-sys\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540616 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-lib-modules\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540819 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-podres\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540819 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-proc\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540819 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540698 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-sys\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540819 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf9fn\" (UniqueName: \"kubernetes.io/projected/3a0418c3-cecc-47a3-8b60-96c14e793ada-kube-api-access-qf9fn\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540948 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-proc\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540948 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540832 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-podres\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.540948 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.540832 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3a0418c3-cecc-47a3-8b60-96c14e793ada-lib-modules\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.549340 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.549310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf9fn\" (UniqueName: \"kubernetes.io/projected/3a0418c3-cecc-47a3-8b60-96c14e793ada-kube-api-access-qf9fn\") pod \"perf-node-gather-daemonset-8prlh\" (UID: \"3a0418c3-cecc-47a3-8b60-96c14e793ada\") " pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.690669 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.690639 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:11.827864 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:11.825893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh"] Jul 23 10:04:11.832036 ip-10-0-132-192 kubenswrapper[2562]: W0723 10:04:11.832012 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3a0418c3_cecc_47a3_8b60_96c14e793ada.slice/crio-d8b6c911dc4c21ed355a4134d0ef2de13b238c90c09ce338664a3f7c32b91973 WatchSource:0}: Error finding container d8b6c911dc4c21ed355a4134d0ef2de13b238c90c09ce338664a3f7c32b91973: Status 404 returned error can't find the container with id d8b6c911dc4c21ed355a4134d0ef2de13b238c90c09ce338664a3f7c32b91973 Jul 23 10:04:12.803992 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:12.803959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" event={"ID":"3a0418c3-cecc-47a3-8b60-96c14e793ada","Type":"ContainerStarted","Data":"a2dd79e78efbd0f2389f7c9532f314fadfd191d21ab64ed9a4f9103bea29058b"} Jul 23 10:04:12.804399 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:12.804008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" event={"ID":"3a0418c3-cecc-47a3-8b60-96c14e793ada","Type":"ContainerStarted","Data":"d8b6c911dc4c21ed355a4134d0ef2de13b238c90c09ce338664a3f7c32b91973"} Jul 23 10:04:12.804399 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:12.804098 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:12.821441 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:12.821387 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" podStartSLOduration=1.821371809 podStartE2EDuration="1.821371809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 10:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 10:04:12.819669672 +0000 UTC m=+2833.029852821" watchObservedRunningTime="2026-07-23 10:04:12.821371809 +0000 UTC m=+2833.031554958" Jul 23 10:04:14.333239 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:14.333208 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cvxp6_c56473cc-da76-4b86-b727-322ff452df75/dns/0.log" Jul 23 10:04:14.352689 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:14.352669 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cvxp6_c56473cc-da76-4b86-b727-322ff452df75/kube-rbac-proxy/0.log" Jul 23 10:04:14.416707 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:14.416679 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7fjf_1ea0c7a2-8481-4f90-a089-31f955dac3b8/dns-node-resolver/0.log" Jul 23 10:04:15.065927 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:15.065882 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mghn8_c85df748-f562-4ed4-a0c9-df107029989c/node-ca/0.log" Jul 23 10:04:16.013688 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:16.013658 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bt5dh_9c15706b-bbba-4497-8153-db9f3dbe9de9/istio-proxy/0.log" Jul 23 10:04:16.144069 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:16.144038 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qzpsq_b00f2c3c-20bb-483a-9d51-624bd6f4990f/discovery/0.log" Jul 23 10:04:16.197250 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:16.197221 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dccdf9c4-8bfm9_66a53353-2dc0-48ca-b973-82d6d050f1cb/kube-auth-proxy/0.log" Jul 23 10:04:16.224024 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:16.223977 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/0.log" Jul 23 10:04:16.250147 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:16.250113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/1.log" Jul 23 10:04:17.078406 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:17.078357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lgmrh_300ce848-6c8c-4028-b5fd-3d4b4d5fad21/serve-healthcheck-canary/0.log" Jul 23 10:04:17.680782 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:17.680755 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kj2j4_801082ef-11db-4f35-86fc-3806e65015cf/kube-rbac-proxy/0.log" Jul 23 10:04:17.702965 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:17.702941 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kj2j4_801082ef-11db-4f35-86fc-3806e65015cf/exporter/0.log" Jul 23 10:04:17.723580 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:17.723561 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kj2j4_801082ef-11db-4f35-86fc-3806e65015cf/extractor/0.log" Jul 23 10:04:18.818746 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:18.818717 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qcmqp/perf-node-gather-daemonset-8prlh" Jul 23 10:04:20.023319 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:20.023295 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-kzdd9_936103be-6c95-4ee3-a6ee-0abd6aa2ebe2/manager/0.log" Jul 23 10:04:20.202807 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:20.202770 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hbdvn_b5c50541-6b3f-4fa0-99fa-f6671d6a8c5b/manager/0.log" Jul 23 10:04:20.500053 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:20.500023 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pr4jf_075a3e0d-dc49-43fa-ad2f-16aedc544bb8/manager/0.log" Jul 23 10:04:20.673880 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:20.673854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d6699bc9-nb66c_718ba171-fc11-4ef7-98bc-774a0d45ebc5/manager/0.log" Jul 23 10:04:20.696099 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:20.696020 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d6699bc9-nb66c_718ba171-fc11-4ef7-98bc-774a0d45ebc5/copy-manifests/0.log" Jul 23 10:04:21.919405 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:21.919363 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bt5dh_9c15706b-bbba-4497-8153-db9f3dbe9de9/istio-proxy/0.log" Jul 23 10:04:22.035864 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:22.035825 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qzpsq_b00f2c3c-20bb-483a-9d51-624bd6f4990f/discovery/0.log" Jul 23 10:04:22.060785 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:22.060747 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dccdf9c4-8bfm9_66a53353-2dc0-48ca-b973-82d6d050f1cb/kube-auth-proxy/0.log" Jul 23 10:04:22.077401 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:22.077372 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/0.log" Jul 23 10:04:22.101842 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:22.101815 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/1.log" Jul 23 10:04:22.108252 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:22.108232 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sxnkf_beaa8acc-e372-42bf-89b2-f9204e38903a/openshift-lws-operator/0.log" Jul 23 10:04:23.470240 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:23.470152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bt5dh_9c15706b-bbba-4497-8153-db9f3dbe9de9/istio-proxy/0.log" Jul 23 10:04:23.588495 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:23.588446 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qzpsq_b00f2c3c-20bb-483a-9d51-624bd6f4990f/discovery/0.log" Jul 23 10:04:23.622315 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:23.622289 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dccdf9c4-8bfm9_66a53353-2dc0-48ca-b973-82d6d050f1cb/kube-auth-proxy/0.log" Jul 23 10:04:23.642311 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:23.642272 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/0.log" Jul 23 10:04:23.667107 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:23.667074 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kvl44_f0af4e63-1e3a-4400-8677-4f2e3bac89ac/istio-proxy/1.log" Jul 23 10:04:27.230102 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:27.230075 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-j27xg_4a245755-f466-47aa-af41-2eb04a26f9ed/migrator/0.log" Jul 23 10:04:27.250278 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:27.250248 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-j27xg_4a245755-f466-47aa-af41-2eb04a26f9ed/graceful-termination/0.log" Jul 23 10:04:28.879954 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:28.879906 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7pgwl_8301271b-a788-4a0b-ac03-386008009cdf/kube-multus/0.log" Jul 23 10:04:28.917120 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:28.917086 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/kube-multus-additional-cni-plugins/0.log" Jul 23 10:04:28.944142 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:28.944113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/egress-router-binary-copy/0.log" Jul 23 10:04:28.968969 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:28.968945 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/cni-plugins/0.log" Jul 23 10:04:28.992553 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:28.992529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/bond-cni-plugin/0.log" Jul 23 10:04:29.016523 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:29.016486 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/routeoverride-cni/0.log" Jul 23 10:04:29.040276 ip-10-0-132-192 kubenswrapper[2562]: I0723 10:04:29.040248 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9fnfr_46e663c0-2d50-4c18-987a-50787a03dd16/whereabouts-cni-bincopy/0.log"