Jul 06 19:17:46.186414 ip-10-0-130-239 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 19:17:46.186427 ip-10-0-130-239 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 19:17:46.186437 ip-10-0-130-239 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 19:17:46.186752 ip-10-0-130-239 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 19:17:56.238770 ip-10-0-130-239 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 19:17:56.238793 ip-10-0-130-239 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c5c5836d9394472da47d06c93f7ead8d -- Jul 06 19:19:43.467301 ip-10-0-130-239 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.843803 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848919 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848932 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848952 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848956 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848961 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848965 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848969 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:43.941177 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848973 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:43.913877 ip-10-0-130-239 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848977 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848981 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848994 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.848999 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849003 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849008 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849014 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849020 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849024 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849029 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849033 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849037 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849041 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849046 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849050 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849054 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849059 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849063 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:43.942448 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849068 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849072 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849076 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849080 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849084 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849088 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849092 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849096 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849099 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849103 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849107 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849111 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849114 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849118 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849122 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849126 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849130 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849134 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849138 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849141 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:43.943996 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849145 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849148 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849152 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849156 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849167 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849171 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849175 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849180 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849184 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849188 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849192 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849196 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849200 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849204 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849208 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849212 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849218 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849223 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849231 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:43.944767 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849237 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849242 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849246 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849251 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849254 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849258 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849263 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849267 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849271 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849275 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849278 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849282 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849285 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849289 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849293 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849297 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849301 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849304 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849308 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849312 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:43.946348 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849317 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849321 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849325 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849329 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849333 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849337 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849341 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849345 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849348 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849352 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849356 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849359 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849363 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.849367 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851714 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851724 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851730 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851734 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851738 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851745 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:43.947145 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851748 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851754 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851761 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851765 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851770 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851774 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851777 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851781 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851785 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851789 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851794 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851798 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851802 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851805 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851809 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851813 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851817 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851821 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851825 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851829 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:43.948699 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851832 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851836 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851840 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851844 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851848 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851852 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851856 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851859 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851863 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851867 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851871 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851875 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851879 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851883 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851887 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851892 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851895 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851899 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851904 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851907 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:43.949512 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851911 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851917 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851923 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851928 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851932 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851951 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851955 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851960 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851963 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851967 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851972 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851976 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851980 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851984 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851988 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851992 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851995 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.851999 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852003 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852007 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:43.951086 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852011 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852015 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852019 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852023 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852026 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852031 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852035 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852039 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852043 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852047 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852051 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852054 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852058 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852062 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852066 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852070 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852074 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852079 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852082 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852086 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:43.951855 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852089 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852097 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852102 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852107 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852112 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852115 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852119 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852123 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852127 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852131 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852134 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852138 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852142 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852755 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852763 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852768 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852772 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852777 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852783 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852787 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:43.954216 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852791 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852794 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852798 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852802 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852805 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852809 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852816 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852820 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852825 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852830 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852835 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852839 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852843 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852847 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852850 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852854 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852858 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852862 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852865 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:43.955001 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852870 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852874 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852878 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852881 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852885 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852889 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852893 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852896 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852900 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852904 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852907 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852912 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852916 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852919 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852923 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852928 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852932 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852950 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852955 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852960 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:43.955778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852964 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852968 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852974 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852978 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852982 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852986 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852989 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852993 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.852997 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853000 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853004 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853008 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853011 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853015 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853019 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853022 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853026 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853030 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853034 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853038 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:43.956566 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853042 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853046 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853051 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853058 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853063 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853068 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853071 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853075 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853079 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853083 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853086 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853092 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853097 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853100 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853104 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853108 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853114 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853117 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853121 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853125 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:43.958147 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853128 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853132 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853137 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853142 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853146 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853151 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853155 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853159 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853163 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853167 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853170 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853174 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.853177 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853258 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853267 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853275 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853281 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853288 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853293 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853299 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853305 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:19:43.958908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853310 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853315 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853322 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853327 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853331 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853336 2576 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853340 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853346 2576 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853351 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853355 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853362 2576 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853366 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853371 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853376 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853381 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853386 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853390 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853395 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853400 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853404 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853408 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853413 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853417 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853422 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:19:43.960460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853428 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853432 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853436 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853441 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853445 2576 flags.go:64] FLAG: --enable-server="true" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853450 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853456 2576 flags.go:64] FLAG: --event-burst="100" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853460 2576 flags.go:64] FLAG: --event-qps="50" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853465 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853469 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853475 2576 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853481 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853485 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853490 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853494 2576 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853510 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853515 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853519 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853524 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853528 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853532 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853536 2576 flags.go:64] FLAG: --feature-gates="" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853542 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853546 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853551 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:19:43.961241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853555 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853561 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853565 2576 flags.go:64] FLAG: --help="false" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853569 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-130-239.ec2.internal" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853574 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853578 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853583 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853588 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853593 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853597 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853602 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853606 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853610 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853623 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853629 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853633 2576 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853670 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853677 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853681 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853685 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853688 2576 flags.go:64] FLAG: --lock-file="" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853692 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853695 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853699 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:19:43.962799 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853715 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853720 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853725 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853728 2576 flags.go:64] FLAG: --logging-format="text" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853731 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853735 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853738 2576 flags.go:64] FLAG: --manifest-url="" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853741 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853746 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853750 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853754 2576 flags.go:64] FLAG: --max-pods="110" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853757 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853760 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853763 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853766 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853770 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853772 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853775 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853782 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853785 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853790 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853794 2576 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853799 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:19:43.963578 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853806 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853809 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853814 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853817 2576 flags.go:64] FLAG: --port="10250" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853820 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853823 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b64f6bd700706c1b" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853826 2576 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853830 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853833 2576 flags.go:64] FLAG: --register-node="true" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853836 2576 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853840 2576 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853843 2576 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853846 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853849 2576 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853853 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853856 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853859 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853862 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853865 2576 flags.go:64] FLAG: --runonce="false" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853870 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853875 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853879 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853882 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853885 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853888 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853891 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:19:43.965168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853894 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853897 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853900 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853903 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853906 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853909 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853912 2576 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853915 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853921 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853925 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853927 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853933 2576 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853951 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853955 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853961 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853964 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853967 2576 flags.go:64] FLAG: --v="2" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853972 2576 flags.go:64] FLAG: --version="false" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853976 2576 flags.go:64] FLAG: --vmodule="" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853980 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.853983 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854130 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854134 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854138 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:43.965979 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854142 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854145 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854147 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854150 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854153 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854156 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854159 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854162 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854165 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854167 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854171 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854175 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854180 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854185 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854187 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854190 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854193 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854196 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854199 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854202 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:43.967574 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854205 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854207 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854211 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854214 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854217 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854219 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854222 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854224 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854227 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854233 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854236 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854239 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854242 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854244 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854247 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854251 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854255 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854259 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854263 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:43.968351 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854265 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854268 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854271 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854273 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854278 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854280 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854283 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854285 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854288 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854291 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854293 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854296 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854305 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854308 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854310 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854314 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854316 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854319 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854322 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854327 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:43.969532 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854332 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854337 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854342 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854345 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854347 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854350 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854353 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854355 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854358 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854360 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854362 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854365 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854368 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854370 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854372 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854376 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854383 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854386 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854389 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854391 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:43.970311 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854394 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854397 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854399 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854402 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854407 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854424 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854428 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854432 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854435 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854438 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854440 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854443 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854446 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854448 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854451 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854453 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:43.970827 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.854456 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.855063 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.861865 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.861879 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.861899 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.861904 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.861988 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.861994 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.861998 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862001 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862004 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862007 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862011 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862014 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:44.061592 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862017 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862020 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862023 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862026 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862028 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862031 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862033 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862037 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862039 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862042 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862045 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862047 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862050 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862053 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862057 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862061 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862064 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862067 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862070 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:44.062143 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862073 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862076 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862079 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862082 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862085 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862088 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862090 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862094 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862096 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862099 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862102 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862105 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862107 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862110 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862112 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862115 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862117 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862120 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862122 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862125 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:44.062747 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862128 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862131 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862134 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862136 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862139 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862141 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862148 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862151 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862153 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862156 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862159 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862162 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862164 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862169 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862172 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862174 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862177 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862180 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862182 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862185 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:44.063371 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862187 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862190 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862193 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862195 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862198 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862200 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862203 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862205 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862208 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862211 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862213 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862216 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862219 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862221 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862224 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862227 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862229 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862232 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862235 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862238 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:44.064197 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862240 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862243 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862245 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862248 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862250 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862254 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862258 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862262 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862267 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862270 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862273 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862276 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.862281 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862398 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:19:44.066683 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862403 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862406 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862409 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862412 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862415 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862418 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862421 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862423 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862426 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862429 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862432 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862435 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862437 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862440 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862443 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862446 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862449 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862452 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862455 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862457 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:19:44.067138 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862460 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862462 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862465 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862467 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862471 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862473 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862476 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862479 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862481 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862484 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862486 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862489 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862491 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862493 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862496 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862499 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862501 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862503 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862506 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862509 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:19:44.067731 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862511 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862514 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862516 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862518 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862521 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862524 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862527 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862529 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862533 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862535 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862538 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862540 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862543 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862545 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862547 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862550 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862553 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862556 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862558 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862561 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:19:44.068332 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862563 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862566 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862569 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862573 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862576 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862580 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862583 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862586 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862589 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862591 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862594 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862597 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862600 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862602 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862606 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862609 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862612 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862614 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:19:44.068876 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862617 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862619 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862623 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862626 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862628 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862631 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862633 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862636 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862638 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862641 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862644 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862647 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862649 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862652 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862654 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862657 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862659 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862661 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862664 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:19:44.069426 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:43.862667 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.862671 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.862769 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.864634 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.866862 2576 server.go:1019] "Starting client certificate rotation" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.866983 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.867048 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.871337 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.873084 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.880803 2576 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.884714 2576 log.go:25] "Validated CRI v1 image API" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.885476 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:19:44.069986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.889356 2576 fs.go:135] Filesystem UUIDs: map[6c45f656-a6c8-4386-a954-5259f7f39122:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e4c58b4e-0d61-45af-b25d-28286135bf54:/dev/nvme0n1p3] Jul 06 19:19:44.070358 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.889368 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.894207 2576 manager.go:216] Machine: {Timestamp:2026-07-06 19:19:43.892921498 +0000 UTC m=+0.310149611 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100113 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228e84c3fa305d1ba7c1b39e1f5f3b SystemUUID:ec228e84-c3fa-305d-1ba7-c1b39e1f5f3b BootID:c5c5836d-9394-472d-a47d-06c93f7ead8d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:07:7f:db:fd:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:07:7f:db:fd:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:53:3b:08:8c:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.894308 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.894379 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897011 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897040 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-239.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897574 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:19:44.070454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897757 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897769 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.897777 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.898336 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.899377 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.901868 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902002 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902032 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902045 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902055 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902064 2576 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902072 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902445 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.902464 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.904433 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.908108 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.908146 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909315 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909329 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909335 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909341 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909349 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909355 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909363 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909368 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909375 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909380 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909393 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.909398 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.910064 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.910071 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.913067 2576 server.go:1268] "Started kubelet" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.913133 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.913145 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.913214 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.914320 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.915053 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.915087 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.915107 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-239.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:19:44.070733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.915412 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.918660 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.919775 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-239.ec2.internal.18bfc98cc5cb3e1a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-239.ec2.internal,UID:ip-10-0-130-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-239.ec2.internal,},FirstTimestamp:2026-07-06 19:19:43.913041434 +0000 UTC m=+0.330269545,LastTimestamp:2026-07-06 19:19:43.913041434 +0000 UTC m=+0.330269545,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-239.ec2.internal,}" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924133 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924149 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.924318 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924720 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924729 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924746 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924835 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.924846 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925357 2576 factory.go:153] Registering CRI-O factory Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925375 2576 factory.go:223] Registration of the crio container factory successfully Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925427 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925436 2576 factory.go:55] Registering systemd factory Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925445 2576 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925466 2576 factory.go:103] Registering Raw factory Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.925497 2576 manager.go:1196] Started watching for new ooms in manager Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.926058 2576 manager.go:349] Starting recovery of all containers Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.937985 2576 manager.go:354] Recovery completed Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.942404 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.967530 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-42pgl" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.973689 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-42pgl" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980266 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980289 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980298 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.980648 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:43.980726 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980837 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980848 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980851 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.980867 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:19:44.073266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981471 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981494 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981511 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981973 2576 policy_none.go:49] "None policy: Start" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981986 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.981998 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:43.982993 2576 policy_none.go:47] "Start" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.024539 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.040627 2576 manager.go:337] "Starting Device Plugin manager" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.040661 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.040671 2576 server.go:87] "Starting device plugin registration server" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.040914 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.040930 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.041000 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.041095 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.041116 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.041586 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:19:44.093031 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.041636 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.104660 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.105793 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.105829 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.105853 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.105894 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.108413 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.141444 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.142125 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.142144 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.142153 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.142172 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-239.ec2.internal" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:44.151479 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-239.ec2.internal" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.151496 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-239.ec2.internal\": node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:44.204529 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.173250 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.063962 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:44.274272 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.181996 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.111672 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.294776 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.211956 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.400372 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.312149 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.412356 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.412337 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.476418 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.476372 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal"] Jul 06 19:19:46.476547 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.476483 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:46.477680 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.477666 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:46.477752 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.477694 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:46.477752 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.477704 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:46.478962 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.478950 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:46.479063 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.479049 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.479111 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.479078 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:46.481078 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481062 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:46.481179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481095 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:46.481179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481109 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:46.481179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481065 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:46.481179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481169 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:46.481179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.481181 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:46.482340 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.482325 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.482417 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.482355 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:19:46.483055 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.483039 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:19:46.483124 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.483066 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:19:46.483124 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.483077 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:19:46.505697 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.505676 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-239.ec2.internal\" not found" node="ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.510265 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.510249 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-239.ec2.internal\" not found" node="ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.512571 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.512559 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.576679 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.576654 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.576784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.576697 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.576784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.576715 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1115f28ae51a4317d8862e47dd2c12eb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-239.ec2.internal\" (UID: \"1115f28ae51a4317d8862e47dd2c12eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.603421 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.603403 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:19:46.603526 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.603510 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:19:46.603630 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.603554 2576 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a8e5f60cb43d34688bcaa27399966a2d-c42927098d644589.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-239.ec2.internal.18bfc98cc9cd2b58\": read tcp 10.0.130.239:54038->100.57.137.91:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-239.ec2.internal.18bfc98cc9cd2b58 default 5788 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-239.ec2.internal,UID:ip-10-0-130-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-239.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-239.ec2.internal,},FirstTimestamp:2026-07-06 19:19:43 +0000 UTC,LastTimestamp:2026-07-06 19:19:46.483053931 +0000 UTC m=+1.163087722,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-239.ec2.internal,}" Jul 06 19:19:46.612648 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.612621 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.677151 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.677151 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.677151 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677144 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1115f28ae51a4317d8862e47dd2c12eb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-239.ec2.internal\" (UID: \"1115f28ae51a4317d8862e47dd2c12eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.677304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677185 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.677304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677233 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1115f28ae51a4317d8862e47dd2c12eb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-239.ec2.internal\" (UID: \"1115f28ae51a4317d8862e47dd2c12eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.677304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.677258 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1089697870923d03089f4f6fa189bd3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal\" (UID: \"b1089697870923d03089f4f6fa189bd3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.712893 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:46.712870 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-239.ec2.internal\" not found" Jul 06 19:19:46.713961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.713941 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:14:45 +0000 UTC" deadline="2028-04-19 13:52:03.654971766 +0000 UTC" Jul 06 19:19:46.714002 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.713961 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15666h32m16.941013008s" Jul 06 19:19:46.734188 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.734163 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:19:46.754075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.754051 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:19:46.762536 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.762518 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.796668 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.796648 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:19:46.798216 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.798203 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.808796 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.808778 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.812686 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.812671 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" Jul 06 19:19:46.813596 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.813582 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:19:46.899187 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:46.899164 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:19:47.354323 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:47.354291 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1089697870923d03089f4f6fa189bd3.slice/crio-6fa417a3fa913c10519efb2260e6a1e3de249dff237a0ec323be6d2f36b0d47b WatchSource:0}: Error finding container 6fa417a3fa913c10519efb2260e6a1e3de249dff237a0ec323be6d2f36b0d47b: Status 404 returned error can't find the container with id 6fa417a3fa913c10519efb2260e6a1e3de249dff237a0ec323be6d2f36b0d47b Jul 06 19:19:47.355877 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:47.355855 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1115f28ae51a4317d8862e47dd2c12eb.slice/crio-5638b69d122b654a3a9c6f487ad50c79f9fef69d93e3d3506a86f27b6b3742f2 WatchSource:0}: Error finding container 5638b69d122b654a3a9c6f487ad50c79f9fef69d93e3d3506a86f27b6b3742f2: Status 404 returned error can't find the container with id 5638b69d122b654a3a9c6f487ad50c79f9fef69d93e3d3506a86f27b6b3742f2 Jul 06 19:19:47.360936 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.360915 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:19:47.640032 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.639956 2576 apiserver.go:52] "Watching apiserver" Jul 06 19:19:47.648784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.648761 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:19:47.649806 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.649786 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4","openshift-cluster-node-tuning-operator/tuned-v5dsq","openshift-multus/multus-additional-cni-plugins-t2vpz","openshift-multus/multus-b46bg","openshift-multus/network-metrics-daemon-68mm4","openshift-network-diagnostics/network-check-target-s2wf7","kube-system/global-pull-secret-syncer-lr9nc","openshift-dns/node-resolver-vvh5p","openshift-image-registry/node-ca-k5rc7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal","openshift-network-operator/iptables-alerter-2kvzr","openshift-ovn-kubernetes/ovnkube-node-pl2cw","kube-system/konnectivity-agent-w76qj"] Jul 06 19:19:47.653413 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.653397 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.656163 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656143 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.656411 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656388 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:19:47.656411 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656392 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:19:47.656552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656421 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.656552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656453 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mr77v\"" Jul 06 19:19:47.656552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.656496 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.658657 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.658640 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5ccmb\"" Jul 06 19:19:47.658846 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.658830 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:19:47.658965 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.658945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.659063 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.659049 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:19:47.661053 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.661033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.662404 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.662384 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:19:47.663245 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663226 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:47.663336 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.663297 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:47.663646 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663631 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:19:47.663714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663634 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:19:47.663808 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663792 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.663856 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663842 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gbqtl\"" Jul 06 19:19:47.663979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663964 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.664024 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.663969 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wbqhz\"" Jul 06 19:19:47.665868 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.665854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.665954 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.665903 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:47.667994 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.667980 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.668081 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.668064 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:47.670229 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.670211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.672385 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.672369 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.672686 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.672650 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xbppp\"" Jul 06 19:19:47.672787 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.672770 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.672874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.672826 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.674454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.674430 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.674613 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.674595 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:19:47.674805 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.674779 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vstc2\"" Jul 06 19:19:47.674896 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.674861 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.674896 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.674871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.675654 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.675390 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:19:47.678665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.678645 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.678772 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.678667 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.678772 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.678671 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:19:47.678850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.678803 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hsvq2\"" Jul 06 19:19:47.679114 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.679099 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.680999 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.680979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-kubernetes\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-system-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681023 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-k8s-cni-cncf-io\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-daemon-config\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681058 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39112394-12d7-400c-a71c-921122fd5f0f-agent-certs\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.681272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681080 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-host\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-os-release\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.681272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-bin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681249 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-multus\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-hostroot\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-run\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-tmp\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681369 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.681420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-socket-dir-parent\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39112394-12d7-400c-a71c-921122fd5f0f-konnectivity-ca\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-conf\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681535 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-systemd\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-cnibin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681585 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-cni-binary-copy\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681606 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-conf-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681643 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ng76\" (UniqueName: \"kubernetes.io/projected/ccb0abf7-182c-4314-9e1f-873c686bf976-kube-api-access-8ng76\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-sys\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681758 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681767 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-os-release\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysconfig\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681806 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681818 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681827 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-lib-modules\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-tuned\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681875 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8c76\" (UniqueName: \"kubernetes.io/projected/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-kube-api-access-p8c76\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-system-cni-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.681921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-netns\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681942 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-kubelet\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-modprobe-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.681992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-var-lib-kubelet\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cnibin\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682058 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-binary-copy\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682097 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682125 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bw7h\" (UniqueName: \"kubernetes.io/projected/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-kube-api-access-5bw7h\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-multus-certs\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-etc-kubernetes\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.682569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.683018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682651 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:19:47.683018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682726 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:19:47.683018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682810 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n6tvk\"" Jul 06 19:19:47.683018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.682863 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:19:47.684433 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.684418 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:19:47.684518 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.684438 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:19:47.684749 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.684728 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gfbt5\"" Jul 06 19:19:47.685439 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.685423 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:19:47.708817 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.708801 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw6nq" Jul 06 19:19:47.718530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.718514 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw6nq" Jul 06 19:19:47.762909 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.762881 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:19:47.782585 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782561 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-config\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.782701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-script-lib\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.782701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-socket-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.782701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-device-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.782701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.782701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782682 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-dbus\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-systemd-units\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-kubernetes\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782791 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j6w2\" (UniqueName: \"kubernetes.io/projected/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-kube-api-access-2j6w2\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-bin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782821 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-kubernetes\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782844 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-multus\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782868 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-hostroot\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782894 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-bin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782902 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-cni-multus\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-run\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.782950 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782936 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-hostroot\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782957 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-tmp\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782969 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-run\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.782985 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpljf\" (UniqueName: \"kubernetes.io/projected/f10cb92d-caa2-4921-afb1-e06ecb497665-kube-api-access-zpljf\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783012 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-socket-dir-parent\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-env-overrides\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783050 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-socket-dir-parent\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-cnibin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdlmk\" (UniqueName: \"kubernetes.io/projected/24b96433-844c-41a0-9cef-dacde28c440f-kube-api-access-zdlmk\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7e4997e-01f4-4909-a2a6-3eee839cf20b-serviceca\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783165 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783177 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-cnibin\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-registration-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783212 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-os-release\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783219 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:19:47.783452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-tuned\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8c76\" (UniqueName: \"kubernetes.io/projected/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-kube-api-access-p8c76\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783308 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-sys-fs\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783309 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-os-release\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783333 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf9bedb2-db52-4c8a-96f3-2834416a1c76-hosts-file\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-kubelet\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-var-lib-kubelet\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-log-socket\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783438 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-var-lib-kubelet\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783447 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bw7h\" (UniqueName: \"kubernetes.io/projected/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-kube-api-access-5bw7h\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783441 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783472 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-kubelet\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783513 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-var-lib-kubelet\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.784156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783639 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-node-log\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9bedb2-db52-4c8a-96f3-2834416a1c76-tmp-dir\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-multus-certs\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783737 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-etc-kubernetes\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-var-lib-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-ovn\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783797 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f10cb92d-caa2-4921-afb1-e06ecb497665-ovn-node-metrics-cert\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783804 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-multus-certs\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-slash\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783806 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-etc-kubernetes\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-system-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783882 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-k8s-cni-cncf-io\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-daemon-config\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39112394-12d7-400c-a71c-921122fd5f0f-agent-certs\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.784830 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783955 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-system-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-host\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.783977 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-k8s-cni-cncf-io\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784021 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-host\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-os-release\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-etc-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784287 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-os-release\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-systemd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qd5\" (UniqueName: \"kubernetes.io/projected/cf9bedb2-db52-4c8a-96f3-2834416a1c76-kube-api-access-v6qd5\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784453 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784472 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-host-slash\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.785563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39112394-12d7-400c-a71c-921122fd5f0f-konnectivity-ca\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784627 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-iptables-alerter-script\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-conf\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-systemd\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784721 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-cni-binary-copy\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784776 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-conf-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ng76\" (UniqueName: \"kubernetes.io/projected/ccb0abf7-182c-4314-9e1f-873c686bf976-kube-api-access-8ng76\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784845 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-systemd\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-sys\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784884 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-conf\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784933 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysctl-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784956 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-conf-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785162 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39112394-12d7-400c-a71c-921122fd5f0f-konnectivity-ca\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.784656 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-daemon-config\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785301 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-sys\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-system-cni-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785400 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ccb0abf7-182c-4314-9e1f-873c686bf976-cni-binary-copy\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-system-cni-dir\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysconfig\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785477 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-lib-modules\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785487 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-sysconfig\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cnibin\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785525 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-binary-copy\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785569 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cnibin\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.786973 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-netns\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wds\" (UniqueName: \"kubernetes.io/projected/e07c6592-51e7-4194-87de-15bf20da8e34-kube-api-access-96wds\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-netns\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-modprobe-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785677 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-netd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcn2\" (UniqueName: \"kubernetes.io/projected/b7e4997e-01f4-4909-a2a6-3eee839cf20b-kube-api-access-brcn2\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-kubelet-config\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7e4997e-01f4-4909-a2a6-3eee839cf20b-host\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785772 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-bin\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.785554 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-lib-modules\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-host-run-netns\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786171 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-modprobe-d\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786287 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ccb0abf7-182c-4314-9e1f-873c686bf976-multus-cni-dir\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786501 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-tmp\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.787423 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786574 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-etc-tuned\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.787795 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786757 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-binary-copy\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.787795 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786768 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.787795 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.786827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39112394-12d7-400c-a71c-921122fd5f0f-agent-certs\") pod \"konnectivity-agent-w76qj\" (UID: \"39112394-12d7-400c-a71c-921122fd5f0f\") " pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.790935 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.790917 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:19:47.790935 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.790936 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:19:47.791050 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.790945 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:47.791050 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.791035 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:48.291021676 +0000 UTC m=+2.971055450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:47.792113 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.792083 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8c76\" (UniqueName: \"kubernetes.io/projected/17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36-kube-api-access-p8c76\") pod \"tuned-v5dsq\" (UID: \"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36\") " pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.792113 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.792096 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bw7h\" (UniqueName: \"kubernetes.io/projected/1b9094bf-9ca0-459f-9b2b-9f3816c2eb89-kube-api-access-5bw7h\") pod \"multus-additional-cni-plugins-t2vpz\" (UID: \"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89\") " pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.793040 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.793023 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ng76\" (UniqueName: \"kubernetes.io/projected/ccb0abf7-182c-4314-9e1f-873c686bf976-kube-api-access-8ng76\") pod \"multus-b46bg\" (UID: \"ccb0abf7-182c-4314-9e1f-873c686bf976\") " pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.848081 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.848037 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" event={"ID":"1115f28ae51a4317d8862e47dd2c12eb","Type":"ContainerStarted","Data":"5638b69d122b654a3a9c6f487ad50c79f9fef69d93e3d3506a86f27b6b3742f2"} Jul 06 19:19:47.849686 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.849659 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" event={"ID":"b1089697870923d03089f4f6fa189bd3","Type":"ContainerStarted","Data":"6fa417a3fa913c10519efb2260e6a1e3de249dff237a0ec323be6d2f36b0d47b"} Jul 06 19:19:47.886712 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886678 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdlmk\" (UniqueName: \"kubernetes.io/projected/24b96433-844c-41a0-9cef-dacde28c440f-kube-api-access-zdlmk\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.886815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886725 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7e4997e-01f4-4909-a2a6-3eee839cf20b-serviceca\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.886815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.886815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-registration-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.886980 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886845 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.886980 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886882 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.886980 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886942 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.886980 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886926 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-registration-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.886979 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-sys-fs\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf9bedb2-db52-4c8a-96f3-2834416a1c76-hosts-file\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887052 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-log-socket\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887055 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-sys-fs\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887077 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-kubelet\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887086 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-log-socket\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887115 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-kubelet\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887120 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf9bedb2-db52-4c8a-96f3-2834416a1c76-hosts-file\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887155 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887124 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-node-log\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887174 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-node-log\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9bedb2-db52-4c8a-96f3-2834416a1c76-tmp-dir\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-var-lib-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-ovn\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f10cb92d-caa2-4921-afb1-e06ecb497665-ovn-node-metrics-cert\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887285 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-slash\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887301 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7e4997e-01f4-4909-a2a6-3eee839cf20b-serviceca\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-etc-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887352 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887358 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-ovn\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-systemd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.887427 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf9bedb2-db52-4c8a-96f3-2834416a1c76-tmp-dir\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887457 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887492 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-slash\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.887823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887498 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-etc-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.887495 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:48.38747886 +0000 UTC m=+3.067512639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887532 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-run-systemd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887579 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-var-lib-openvswitch\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887607 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qd5\" (UniqueName: \"kubernetes.io/projected/cf9bedb2-db52-4c8a-96f3-2834416a1c76-kube-api-access-v6qd5\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887631 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-host-slash\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-iptables-alerter-script\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887682 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-host-slash\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-netns\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96wds\" (UniqueName: \"kubernetes.io/projected/e07c6592-51e7-4194-87de-15bf20da8e34-kube-api-access-96wds\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887780 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-netd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887804 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcn2\" (UniqueName: \"kubernetes.io/projected/b7e4997e-01f4-4909-a2a6-3eee839cf20b-kube-api-access-brcn2\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.888563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-kubelet-config\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887851 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7e4997e-01f4-4909-a2a6-3eee839cf20b-host\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-bin\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-config\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.887920 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-script-lib\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.887951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-socket-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:47.887996 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:19:48.387963913 +0000 UTC m=+3.067997695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-device-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888055 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888066 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-socket-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888066 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-run-netns\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-dbus\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-systemd-units\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j6w2\" (UniqueName: \"kubernetes.io/projected/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-kube-api-access-2j6w2\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888182 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpljf\" (UniqueName: \"kubernetes.io/projected/f10cb92d-caa2-4921-afb1-e06ecb497665-kube-api-access-zpljf\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888182 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-kubelet-config\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888194 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-bin\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-env-overrides\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888239 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7e4997e-01f4-4909-a2a6-3eee839cf20b-host\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888259 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e69ee40d-db1c-4469-929c-9338f79a9e97-dbus\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888321 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-device-dir\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888409 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e07c6592-51e7-4194-87de-15bf20da8e34-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888456 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-systemd-units\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888452 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f10cb92d-caa2-4921-afb1-e06ecb497665-host-cni-netd\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888731 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-env-overrides\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888845 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-script-lib\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.888909 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-iptables-alerter-script\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.889730 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.889157 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f10cb92d-caa2-4921-afb1-e06ecb497665-ovnkube-config\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.890220 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.889910 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f10cb92d-caa2-4921-afb1-e06ecb497665-ovn-node-metrics-cert\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.896953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.896928 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdlmk\" (UniqueName: \"kubernetes.io/projected/24b96433-844c-41a0-9cef-dacde28c440f-kube-api-access-zdlmk\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:47.900562 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.900535 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qd5\" (UniqueName: \"kubernetes.io/projected/cf9bedb2-db52-4c8a-96f3-2834416a1c76-kube-api-access-v6qd5\") pod \"node-resolver-vvh5p\" (UID: \"cf9bedb2-db52-4c8a-96f3-2834416a1c76\") " pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.901032 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.901009 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j6w2\" (UniqueName: \"kubernetes.io/projected/d7dbca45-6102-4b18-9265-80a4f1ba7fc1-kube-api-access-2j6w2\") pod \"iptables-alerter-2kvzr\" (UID: \"d7dbca45-6102-4b18-9265-80a4f1ba7fc1\") " pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:47.901498 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.901476 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcn2\" (UniqueName: \"kubernetes.io/projected/b7e4997e-01f4-4909-a2a6-3eee839cf20b-kube-api-access-brcn2\") pod \"node-ca-k5rc7\" (UID: \"b7e4997e-01f4-4909-a2a6-3eee839cf20b\") " pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.901647 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.901625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wds\" (UniqueName: \"kubernetes.io/projected/e07c6592-51e7-4194-87de-15bf20da8e34-kube-api-access-96wds\") pod \"aws-ebs-csi-driver-node-gvqj4\" (UID: \"e07c6592-51e7-4194-87de-15bf20da8e34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:47.902209 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.902188 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpljf\" (UniqueName: \"kubernetes.io/projected/f10cb92d-caa2-4921-afb1-e06ecb497665-kube-api-access-zpljf\") pod \"ovnkube-node-pl2cw\" (UID: \"f10cb92d-caa2-4921-afb1-e06ecb497665\") " pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:47.962615 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.962585 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b46bg" Jul 06 19:19:47.968482 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.968458 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:19:47.975935 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.975762 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" Jul 06 19:19:47.982224 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.982055 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" Jul 06 19:19:47.984556 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:47.984529 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39112394_12d7_400c_a71c_921122fd5f0f.slice/crio-1d2a83bf440edfeb0cf962ab6d5378596c2c77a2c020aa28c8f70bf74c342faa WatchSource:0}: Error finding container 1d2a83bf440edfeb0cf962ab6d5378596c2c77a2c020aa28c8f70bf74c342faa: Status 404 returned error can't find the container with id 1d2a83bf440edfeb0cf962ab6d5378596c2c77a2c020aa28c8f70bf74c342faa Jul 06 19:19:47.987560 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:47.987509 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d09ba5_3ba9_4bc2_a3d2_90f0f4644f36.slice/crio-a885f835f6eb83dfb085324f94470ad9d50e0e5c142e8b6878a6250490cbf467 WatchSource:0}: Error finding container a885f835f6eb83dfb085324f94470ad9d50e0e5c142e8b6878a6250490cbf467: Status 404 returned error can't find the container with id a885f835f6eb83dfb085324f94470ad9d50e0e5c142e8b6878a6250490cbf467 Jul 06 19:19:47.987620 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.987572 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvh5p" Jul 06 19:19:47.992315 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.992299 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5rc7" Jul 06 19:19:47.993607 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:47.993584 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9094bf_9ca0_459f_9b2b_9f3816c2eb89.slice/crio-77d8fd24aa45a6594de688f64438c8331f5a3b8c9f553dac75c9bb8b3f96b3b7 WatchSource:0}: Error finding container 77d8fd24aa45a6594de688f64438c8331f5a3b8c9f553dac75c9bb8b3f96b3b7: Status 404 returned error can't find the container with id 77d8fd24aa45a6594de688f64438c8331f5a3b8c9f553dac75c9bb8b3f96b3b7 Jul 06 19:19:47.998814 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:47.998674 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2kvzr" Jul 06 19:19:48.000752 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:48.000729 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9bedb2_db52_4c8a_96f3_2834416a1c76.slice/crio-677a16f4dcbd0a906ac39762c45bc72771e6b9d9f3532f0e1bd6011f21040378 WatchSource:0}: Error finding container 677a16f4dcbd0a906ac39762c45bc72771e6b9d9f3532f0e1bd6011f21040378: Status 404 returned error can't find the container with id 677a16f4dcbd0a906ac39762c45bc72771e6b9d9f3532f0e1bd6011f21040378 Jul 06 19:19:48.004095 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.003909 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:19:48.007103 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:48.007081 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e4997e_01f4_4909_a2a6_3eee839cf20b.slice/crio-742aa9d1f70a08498206ce828b29ef114cf028306ca7704f579121c057d2c7c3 WatchSource:0}: Error finding container 742aa9d1f70a08498206ce828b29ef114cf028306ca7704f579121c057d2c7c3: Status 404 returned error can't find the container with id 742aa9d1f70a08498206ce828b29ef114cf028306ca7704f579121c057d2c7c3 Jul 06 19:19:48.007889 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.007732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" Jul 06 19:19:48.024077 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:48.024051 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10cb92d_caa2_4921_afb1_e06ecb497665.slice/crio-9e186bb67ee925b0efbea6b515d0413af6e413591b3b098cb7843ede20407adf WatchSource:0}: Error finding container 9e186bb67ee925b0efbea6b515d0413af6e413591b3b098cb7843ede20407adf: Status 404 returned error can't find the container with id 9e186bb67ee925b0efbea6b515d0413af6e413591b3b098cb7843ede20407adf Jul 06 19:19:48.024772 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:19:48.024750 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07c6592_51e7_4194_87de_15bf20da8e34.slice/crio-0cd2bfba90cdde0c5eaf6c63ee7698678ce955c1ec44c5aad2fabc92ac19aedf WatchSource:0}: Error finding container 0cd2bfba90cdde0c5eaf6c63ee7698678ce955c1ec44c5aad2fabc92ac19aedf: Status 404 returned error can't find the container with id 0cd2bfba90cdde0c5eaf6c63ee7698678ce955c1ec44c5aad2fabc92ac19aedf Jul 06 19:19:48.099419 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.099383 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:19:48.391720 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.391688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:48.391877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.391742 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:48.391877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.391780 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:48.391877 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391833 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391890 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391911 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391928 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391898 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:49.391881478 +0000 UTC m=+4.071915265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391940 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391969 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:19:49.391939542 +0000 UTC m=+4.071973317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:48.392025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:48.391985 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:49.391976663 +0000 UTC m=+4.072010438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:48.721619 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.721470 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:14:47 +0000 UTC" deadline="2028-04-16 07:04:09.44884447 +0000 UTC" Jul 06 19:19:48.721619 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.721536 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15587h44m20.727313377s" Jul 06 19:19:48.852833 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.852794 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"9e186bb67ee925b0efbea6b515d0413af6e413591b3b098cb7843ede20407adf"} Jul 06 19:19:48.853999 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.853971 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5rc7" event={"ID":"b7e4997e-01f4-4909-a2a6-3eee839cf20b","Type":"ContainerStarted","Data":"742aa9d1f70a08498206ce828b29ef114cf028306ca7704f579121c057d2c7c3"} Jul 06 19:19:48.855158 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.855120 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerStarted","Data":"77d8fd24aa45a6594de688f64438c8331f5a3b8c9f553dac75c9bb8b3f96b3b7"} Jul 06 19:19:48.856276 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.856248 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" event={"ID":"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36","Type":"ContainerStarted","Data":"a885f835f6eb83dfb085324f94470ad9d50e0e5c142e8b6878a6250490cbf467"} Jul 06 19:19:48.857402 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.857379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2kvzr" event={"ID":"d7dbca45-6102-4b18-9265-80a4f1ba7fc1","Type":"ContainerStarted","Data":"109333d28ac647c7043c8b48d47f55ea1c81590bb55269fedddf06318cfa737d"} Jul 06 19:19:48.858555 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.858533 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvh5p" event={"ID":"cf9bedb2-db52-4c8a-96f3-2834416a1c76","Type":"ContainerStarted","Data":"677a16f4dcbd0a906ac39762c45bc72771e6b9d9f3532f0e1bd6011f21040378"} Jul 06 19:19:48.859621 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.859597 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w76qj" event={"ID":"39112394-12d7-400c-a71c-921122fd5f0f","Type":"ContainerStarted","Data":"1d2a83bf440edfeb0cf962ab6d5378596c2c77a2c020aa28c8f70bf74c342faa"} Jul 06 19:19:48.860622 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.860598 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b46bg" event={"ID":"ccb0abf7-182c-4314-9e1f-873c686bf976","Type":"ContainerStarted","Data":"f840da3bad97a235adf1a98d45b903b473ab516072033ee26d8473309131cf61"} Jul 06 19:19:48.861608 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:48.861585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" event={"ID":"e07c6592-51e7-4194-87de-15bf20da8e34","Type":"ContainerStarted","Data":"0cd2bfba90cdde0c5eaf6c63ee7698678ce955c1ec44c5aad2fabc92ac19aedf"} Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.399570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.399658 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.399692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399823 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399887 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:19:51.399867002 +0000 UTC m=+6.079900791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399902 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399926 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:19:49.399930 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399941 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:49.400481 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399901 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:49.400481 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.399994 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:51.399977544 +0000 UTC m=+6.080011323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:49.400481 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.400070 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:51.400037158 +0000 UTC m=+6.080070956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:49.721909 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.721867 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:14:47 +0000 UTC" deadline="2028-03-16 01:38:38.930293604 +0000 UTC" Jul 06 19:19:49.721909 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.721907 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14838h18m49.20839164s" Jul 06 19:19:49.843832 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.843495 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:49.843832 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.843623 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:49.843832 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.843710 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:49.843832 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.843790 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:49.844446 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:49.844230 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:49.844446 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:49.844345 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:51.416262 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.416224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:51.416733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.416319 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:51.416733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.416360 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:51.416733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.416489 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:51.416733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.416553 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:19:55.416534306 +0000 UTC m=+10.096568084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.416958 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.416982 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.416995 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.417042 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:55.417027224 +0000 UTC m=+10.097061015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.417246 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:51.417340 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.417312 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:19:55.417296326 +0000 UTC m=+10.097330116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:51.844075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.843799 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:51.844075 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.843940 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:51.844311 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.844214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:51.844376 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.844319 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:51.844615 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:51.844479 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:51.844615 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:51.844576 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:53.843314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:53.843676 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:53.843430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:53.843769 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:53.843314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:53.844272 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:53.843854 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:55.449723 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.449629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:55.449723 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.449713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.449745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449826 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449845 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449851 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449864 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449866 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449898 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:20:03.449880953 +0000 UTC m=+18.129914729 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449920 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:03.449903066 +0000 UTC m=+18.129936841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:19:55.450201 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.449939 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:03.449929256 +0000 UTC m=+18.129963049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:19:55.844709 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.844678 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:55.844868 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.844773 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:55.844868 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.844854 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:55.844975 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:55.844927 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:55.845025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.844923 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:55.845025 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:55.845007 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:57.843337 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:57.843306 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:57.843788 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:57.843445 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:57.843860 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:57.843835 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:57.843958 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:57.843934 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:57.844210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:57.844171 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:57.844309 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:57.844279 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:58.888033 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:58.887961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" event={"ID":"17d09ba5-3ba9-4bc2-a3d2-90f0f4644f36","Type":"ContainerStarted","Data":"46ca0a8dc31056b032ccefd49dda19d2f752cfdf4488b7c26cc8003ed7e807f7"} Jul 06 19:19:58.891319 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:58.891288 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" event={"ID":"1115f28ae51a4317d8862e47dd2c12eb","Type":"ContainerStarted","Data":"18d725a7dba45f2d61d63c688c4351d36cd75c6274f874a6ffc8aee6c5138615"} Jul 06 19:19:58.915525 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:58.915478 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v5dsq" podStartSLOduration=3.722582634 podStartE2EDuration="13.915462092s" totalImagesPullingTime="10.192879458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:58.914875315 +0000 UTC m=+13.594909115" watchObservedRunningTime="2026-07-06 19:19:58.915462092 +0000 UTC m=+13.595495889" Jul 06 19:19:59.843563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.843328 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:19:59.843738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.843328 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:19:59.843738 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:59.843686 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:19:59.843869 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:59.843747 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:19:59.843869 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.843335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:19:59.843968 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:19:59.843890 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:19:59.895097 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.895021 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" event={"ID":"e07c6592-51e7-4194-87de-15bf20da8e34","Type":"ContainerStarted","Data":"20f2a45d6bb003aae72ddf9dd4fcb48c05880d0b37d571891458934828763e03"} Jul 06 19:19:59.896644 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.896620 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5rc7" event={"ID":"b7e4997e-01f4-4909-a2a6-3eee839cf20b","Type":"ContainerStarted","Data":"a42eea1dff624a3dab63864cbf972891e6572f6240d0755fb3a453d890cbe739"} Jul 06 19:19:59.900631 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.900608 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerStarted","Data":"9b968be4a87b12384dd9969b0654239eb82346f9024014719076407c1e74d921"} Jul 06 19:19:59.902055 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.902019 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" event={"ID":"b1089697870923d03089f4f6fa189bd3","Type":"ContainerStarted","Data":"39bccd7944282d552458bbc2c02839f685e54ac121d65d5950c112d4e47439db"} Jul 06 19:19:59.904191 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.904168 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2kvzr" event={"ID":"d7dbca45-6102-4b18-9265-80a4f1ba7fc1","Type":"ContainerStarted","Data":"a23b210467094e59bef6c962c8dc7abc38487ac2241987c1f352ddfcd2041676"} Jul 06 19:19:59.905587 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.905565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvh5p" event={"ID":"cf9bedb2-db52-4c8a-96f3-2834416a1c76","Type":"ContainerStarted","Data":"d90eebae716f46208fd024662dcc0013628b931d6a12c9e62612af9254fb3f9f"} Jul 06 19:19:59.906891 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.906860 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w76qj" event={"ID":"39112394-12d7-400c-a71c-921122fd5f0f","Type":"ContainerStarted","Data":"cfa96c0a5fc8d18ac0bf04859bb98dca5a9595b40ba4100abdd0b97eccefd2a8"} Jul 06 19:19:59.916298 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.916257 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k5rc7" podStartSLOduration=3.776095509 podStartE2EDuration="13.916245007s" totalImagesPullingTime="10.140149498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:59.915907217 +0000 UTC m=+14.595941013" watchObservedRunningTime="2026-07-06 19:19:59.916245007 +0000 UTC m=+14.596278805" Jul 06 19:19:59.916641 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.916610 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-239.ec2.internal" podStartSLOduration=13.9166018 podStartE2EDuration="13.9166018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:58.940898429 +0000 UTC m=+13.620932227" watchObservedRunningTime="2026-07-06 19:19:59.9166018 +0000 UTC m=+14.596635596" Jul 06 19:19:59.966035 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:19:59.965987 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2kvzr" podStartSLOduration=3.831264601 podStartE2EDuration="13.965973916s" totalImagesPullingTime="10.134709315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:59.965504912 +0000 UTC m=+14.645538714" watchObservedRunningTime="2026-07-06 19:19:59.965973916 +0000 UTC m=+14.646007713" Jul 06 19:20:00.004537 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:00.004493 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w76qj" podStartSLOduration=4.836645298 podStartE2EDuration="15.004476491s" totalImagesPullingTime="10.167831193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:00.003829381 +0000 UTC m=+14.683863180" watchObservedRunningTime="2026-07-06 19:20:00.004476491 +0000 UTC m=+14.684510288" Jul 06 19:20:00.024148 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:00.024091 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vvh5p" podStartSLOduration=3.866577888 podStartE2EDuration="14.024075569s" totalImagesPullingTime="10.157497681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:00.024009514 +0000 UTC m=+14.704043312" watchObservedRunningTime="2026-07-06 19:20:00.024075569 +0000 UTC m=+14.704109367" Jul 06 19:20:01.843645 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:01.843615 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:01.843645 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:01.843641 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:01.844386 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:01.843747 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:01.844386 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:01.843850 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:01.844386 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:01.843871 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:01.844386 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:01.843936 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:02.916560 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:02.916524 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="9b968be4a87b12384dd9969b0654239eb82346f9024014719076407c1e74d921" exitCode=0 Jul 06 19:20:02.917093 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:02.916599 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"9b968be4a87b12384dd9969b0654239eb82346f9024014719076407c1e74d921"} Jul 06 19:20:02.918238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:02.918208 2576 generic.go:356] "Generic (PLEG): container finished" podID="b1089697870923d03089f4f6fa189bd3" containerID="39bccd7944282d552458bbc2c02839f685e54ac121d65d5950c112d4e47439db" exitCode=0 Jul 06 19:20:02.918371 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:02.918254 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" event={"ID":"b1089697870923d03089f4f6fa189bd3","Type":"ContainerDied","Data":"39bccd7944282d552458bbc2c02839f685e54ac121d65d5950c112d4e47439db"} Jul 06 19:20:03.515408 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.515371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.515453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.515485 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515534 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515562 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515575 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515582 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515585 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:20:03.515629 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515634 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:19.51561601 +0000 UTC m=+34.195649785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:20:03.516001 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515653 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:19.515643935 +0000 UTC m=+34.195677715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:20:03.516001 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.515670 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:20:19.515661343 +0000 UTC m=+34.195695118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:20:03.844388 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.844300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:03.844388 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.844354 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:03.844598 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:03.844314 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:03.844598 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.844439 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:03.844598 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.844533 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:03.844748 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:03.844611 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:04.174751 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:04.174346 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:20:04.175229 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:04.175048 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:20:04.922565 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:04.922535 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:20:04.923144 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:04.923112 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w76qj" Jul 06 19:20:05.844187 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:05.844154 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:05.844607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:05.844233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:05.844607 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:05.844251 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:05.844607 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:05.844361 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:05.844607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:05.844415 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:05.844607 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:05.844495 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:07.844506 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.844323 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:07.844897 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.844323 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:07.844897 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:07.844619 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:07.844897 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.844354 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:07.844897 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:07.844701 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:07.844897 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:07.844737 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:07.933267 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.933043 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"0dea5f6e79b4da0144dbd6255bf16226f8740849eedbabfc9c29139ce84e1650"} Jul 06 19:20:07.933369 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.933346 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"432db2fd7b53979a6d7e7bfa7b57d95799d1724848121fa7eeff161f2d9d5bb4"} Jul 06 19:20:07.934891 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.934868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" event={"ID":"b1089697870923d03089f4f6fa189bd3","Type":"ContainerStarted","Data":"744bee3a6d10343a40b673c06df89836b629399da0a7e5981dfc820955dcfa09"} Jul 06 19:20:07.937829 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.937427 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b46bg" event={"ID":"ccb0abf7-182c-4314-9e1f-873c686bf976","Type":"ContainerStarted","Data":"a0483d8b65cd05bff9382999a6efeb2dfbc407e92b935cf19c26bda8c843c362"} Jul 06 19:20:07.943597 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.943577 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:20:07.959328 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.959293 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-239.ec2.internal" podStartSLOduration=21.959281175 podStartE2EDuration="21.959281175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:07.958992783 +0000 UTC m=+22.639026579" watchObservedRunningTime="2026-07-06 19:20:07.959281175 +0000 UTC m=+22.639314972" Jul 06 19:20:07.977398 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:07.977358 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b46bg" podStartSLOduration=3.473463085 podStartE2EDuration="22.977343611s" totalImagesPullingTime="19.503880526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:07.977003919 +0000 UTC m=+22.657037716" watchObservedRunningTime="2026-07-06 19:20:07.977343611 +0000 UTC m=+22.657377413" Jul 06 19:20:08.789352 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.789256 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:20:07.943595533Z","UUID":"a5a43116-2dd0-4396-9ca9-cab11b47c8f2","Handler":null,"Name":"","Endpoint":""} Jul 06 19:20:08.790995 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.790966 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:20:08.791114 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.791008 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:20:08.941150 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.941102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" event={"ID":"e07c6592-51e7-4194-87de-15bf20da8e34","Type":"ContainerStarted","Data":"87f1ab663582ce09f53a17ec6774944a558d481ac58c2a95e160b0e7221dcbbd"} Jul 06 19:20:08.943794 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.943770 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"ef04b9ee9320596560f3c2ddb7fce924304dec0e40ea89ba68993d35390c44d9"} Jul 06 19:20:08.943927 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.943845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"ac94391ca46c1545ad9849975959c5bd523ce201b44c74eb436e40a355866204"} Jul 06 19:20:08.943927 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.943897 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"9426c4276244b9c28ae3087b962c8c4dd15041529c38b717886b802e68d9d086"} Jul 06 19:20:08.944033 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:08.943974 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"f8a251937a13257d39ee506d5a7787226f9f73b5f59aa5acff8454fe2b611061"} Jul 06 19:20:09.843540 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:09.843515 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:09.843727 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:09.843525 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:09.843727 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:09.843610 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:09.843727 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:09.843686 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:09.843849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:09.843734 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:09.843849 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:09.843778 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:09.947464 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:09.947433 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" event={"ID":"e07c6592-51e7-4194-87de-15bf20da8e34","Type":"ContainerStarted","Data":"9e762fbb770c2c20efa36b5014bd5557141e2f224034c982bdb216a88f57e916"} Jul 06 19:20:09.965931 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:09.965891 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvqj4" podStartSLOduration=4.188254702 podStartE2EDuration="23.965878487s" totalImagesPullingTime="19.777623785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:09.965754611 +0000 UTC m=+24.645788409" watchObservedRunningTime="2026-07-06 19:20:09.965878487 +0000 UTC m=+24.645912283" Jul 06 19:20:10.951918 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:10.951887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"244e5b74283541b52828554b05958db77d5621e9952674a2e8f76437698a0d5e"} Jul 06 19:20:11.844339 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:11.844301 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:11.844523 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:11.844310 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:11.844523 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:11.844451 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:11.844523 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:11.844491 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:11.844680 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:11.844565 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:11.844680 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:11.844647 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:12.959550 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:12.959467 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" event={"ID":"f10cb92d-caa2-4921-afb1-e06ecb497665","Type":"ContainerStarted","Data":"86521e23a6378904d2a62df0422b8d87613123cc2a7ca95e8710649ef2f35a68"} Jul 06 19:20:12.960279 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:12.959729 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:12.975057 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:12.974918 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:12.990554 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:12.990515 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" podStartSLOduration=7.431786555 podStartE2EDuration="26.990502274s" totalImagesPullingTime="19.558715719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:12.990090287 +0000 UTC m=+27.670124082" watchObservedRunningTime="2026-07-06 19:20:12.990502274 +0000 UTC m=+27.670536071" Jul 06 19:20:13.844172 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.844035 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:13.844172 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.844044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:13.844369 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:13.844194 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:13.844369 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.844256 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:13.844455 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:13.844363 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:13.844455 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:13.844443 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:13.962744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.962712 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:13.963274 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.962798 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:13.978649 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:13.978614 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:14.499674 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.499640 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lr9nc"] Jul 06 19:20:14.499857 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.499760 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:14.499900 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:14.499870 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:14.503912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.503868 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-68mm4"] Jul 06 19:20:14.504042 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.503971 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:14.504097 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:14.504075 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:14.507954 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.507895 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2wf7"] Jul 06 19:20:14.508109 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:14.507985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:14.508109 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:14.508076 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:15.844907 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:15.844875 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:15.845320 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:15.844962 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:15.845320 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:15.845046 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:15.845320 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:15.845126 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:15.967524 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:15.967494 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="497e57a9479fbdc15c2895df64e39a8316714a2ff3b5dbbcecdc008589315713" exitCode=0 Jul 06 19:20:15.967674 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:15.967564 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"497e57a9479fbdc15c2895df64e39a8316714a2ff3b5dbbcecdc008589315713"} Jul 06 19:20:16.844092 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:16.844024 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:16.844247 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:16.844126 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:17.844280 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:17.844067 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:17.844623 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:17.844072 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:17.844623 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:17.844398 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:17.844623 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:17.844431 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:17.972643 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:17.972615 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="f88f831866b63e0011462fdabb275a9b79c9024caaa6233e4b92dc3c5bf0e9ea" exitCode=0 Jul 06 19:20:17.972801 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:17.972653 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"f88f831866b63e0011462fdabb275a9b79c9024caaa6233e4b92dc3c5bf0e9ea"} Jul 06 19:20:18.843319 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:18.843286 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:18.843496 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:18.843383 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:19.547676 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.547603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:19.547676 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.547635 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:19.547676 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.547655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547760 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547761 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547778 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547793 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547804 2576 projected.go:196] Error preparing data for projected volume kube-api-access-n8s2v for pod openshift-network-diagnostics/network-check-target-s2wf7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547822 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs podName:24b96433-844c-41a0-9cef-dacde28c440f nodeName:}" failed. No retries permitted until 2026-07-06 19:20:51.547803738 +0000 UTC m=+66.227837527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs") pod "network-metrics-daemon-68mm4" (UID: "24b96433-844c-41a0-9cef-dacde28c440f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547839 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v podName:66da0fd3-366a-45a3-b755-7fcc37389e00 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:51.54782878 +0000 UTC m=+66.227862558 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8s2v" (UniqueName: "kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v") pod "network-check-target-s2wf7" (UID: "66da0fd3-366a-45a3-b755-7fcc37389e00") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:20:19.548276 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.547855 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret podName:e69ee40d-db1c-4469-929c-9338f79a9e97 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:51.547847718 +0000 UTC m=+66.227881507 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret") pod "global-pull-secret-syncer-lr9nc" (UID: "e69ee40d-db1c-4469-929c-9338f79a9e97") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:20:19.844277 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.844211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:19.844399 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.844307 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2wf7" podUID="66da0fd3-366a-45a3-b755-7fcc37389e00" Jul 06 19:20:19.844399 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.844364 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:19.844503 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:19.844483 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-68mm4" podUID="24b96433-844c-41a0-9cef-dacde28c440f" Jul 06 19:20:19.979238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.979211 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="e1cd5df6072994596f4f55f7979e13ee820b0bd8627d7532c6966912bcb4d688" exitCode=0 Jul 06 19:20:19.979383 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:19.979257 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"e1cd5df6072994596f4f55f7979e13ee820b0bd8627d7532c6966912bcb4d688"} Jul 06 19:20:20.843477 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:20.843449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:20.844017 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:20.843576 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lr9nc" podUID="e69ee40d-db1c-4469-929c-9338f79a9e97" Jul 06 19:20:21.113687 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.113606 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-239.ec2.internal" event="NodeReady" Jul 06 19:20:21.113834 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.113765 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:20:21.185972 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.185943 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:20:21.209519 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.209489 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl"] Jul 06 19:20:21.209665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.209619 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.224083 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.224042 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:20:21.224239 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.224095 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:20:21.224351 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.224320 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:20:21.224489 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.224453 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zrfbt\"" Jul 06 19:20:21.225771 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.225740 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8"] Jul 06 19:20:21.225958 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.225935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.239270 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.239247 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j"] Jul 06 19:20:21.239418 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.239366 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" Jul 06 19:20:21.256680 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.256510 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.256914 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.256836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.261559 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.261190 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gdrrs"] Jul 06 19:20:21.262101 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262066 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.262491 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262464 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.262612 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.262612 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.262744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.262744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262634 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.262744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.262744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262708 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gdb7\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.262744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjf2w\" (UniqueName: \"kubernetes.io/projected/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-kube-api-access-wjf2w\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262801 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cbm\" (UniqueName: \"kubernetes.io/projected/9ae5131a-c00c-4876-912f-bba5f0b6281e-kube-api-access-94cbm\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.263019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.262896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rhl\" (UniqueName: \"kubernetes.io/projected/cd0e201a-7fcf-41b4-a2e3-49fae83fba8d-kube-api-access-c9rhl\") pod \"volume-data-source-validator-5d456fdbcd-2hrr8\" (UID: \"cd0e201a-7fcf-41b4-a2e3-49fae83fba8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" Jul 06 19:20:21.267756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.267160 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gzqb5\"" Jul 06 19:20:21.267756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.267385 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.269075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.268838 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 19:20:21.269075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.268884 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.269075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.268922 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.269075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.268932 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wlkwh\"" Jul 06 19:20:21.269075 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.268976 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.269396 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.269224 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:20:21.271198 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.271000 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:20:21.274575 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.274311 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rk7gb\"" Jul 06 19:20:21.277761 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.277742 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:20:21.286581 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.286507 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pl7cl"] Jul 06 19:20:21.286692 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.286645 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.291328 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.291308 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.291546 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.291530 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xzqj7\"" Jul 06 19:20:21.291710 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.291692 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 19:20:21.291770 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.291717 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.292407 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.292384 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 19:20:21.299037 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.299018 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 19:20:21.302531 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.302513 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-696bd7f7cc-8fbvx"] Jul 06 19:20:21.302643 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.302629 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.307856 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.307642 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wvt7m\"" Jul 06 19:20:21.307856 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.307868 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 19:20:21.308149 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.308090 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 19:20:21.311035 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.311014 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.314559 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.314531 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.314640 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.314581 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 19:20:21.330993 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.330972 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx"] Jul 06 19:20:21.331117 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.331100 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.335405 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335278 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 19:20:21.335507 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335474 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.335955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335641 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 19:20:21.335955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335723 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qwzqp\"" Jul 06 19:20:21.335955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335820 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 19:20:21.335955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335860 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.335955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.335940 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 19:20:21.352007 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.351990 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc"] Jul 06 19:20:21.352116 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.352107 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.355581 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.355557 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.355581 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.355572 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.355734 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.355583 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 19:20:21.355734 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.355557 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 19:20:21.356545 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.356526 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t2njv\"" Jul 06 19:20:21.363357 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363335 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.363466 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.363466 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-config\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.363547 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363464 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.363547 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.363494 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:21.363619 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.363554 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:21.863535312 +0000 UTC m=+36.543569098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:21.363619 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363589 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-default-certificate\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.363717 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363624 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrw8w\" (UniqueName: \"kubernetes.io/projected/0763d32a-7979-4dcb-8fff-d5842fa0ee83-kube-api-access-vrw8w\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.363717 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-snapshots\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.363717 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/408aa770-e6eb-4eb9-b635-8e65281f4183-serving-cert\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363718 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjf2w\" (UniqueName: \"kubernetes.io/projected/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-kube-api-access-wjf2w\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363743 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363765 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnxrn\" (UniqueName: \"kubernetes.io/projected/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-kube-api-access-vnxrn\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363785 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363801 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.363850 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cbm\" (UniqueName: \"kubernetes.io/projected/9ae5131a-c00c-4876-912f-bba5f0b6281e-kube-api-access-94cbm\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-tmp\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363958 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-stats-auth\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.363984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.363987 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.364027 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.364071 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.364078 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:21.864060705 +0000 UTC m=+36.544094496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364010 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364119 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.364110 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:21.864097894 +0000 UTC m=+36.544131682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364157 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg9m8\" (UniqueName: \"kubernetes.io/projected/408aa770-e6eb-4eb9-b635-8e65281f4183-kube-api-access-zg9m8\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-serving-cert\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364241 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-trusted-ca\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364333 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364351 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364408 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rhl\" (UniqueName: \"kubernetes.io/projected/cd0e201a-7fcf-41b4-a2e3-49fae83fba8d-kube-api-access-c9rhl\") pod \"volume-data-source-validator-5d456fdbcd-2hrr8\" (UID: \"cd0e201a-7fcf-41b4-a2e3-49fae83fba8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364435 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4vnh\" (UniqueName: \"kubernetes.io/projected/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-kube-api-access-c4vnh\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364465 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364495 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gdb7\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.364682 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-config\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.365112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.364850 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.365445 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.365421 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.366543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366526 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366568 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366589 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366608 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366626 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gdrrs"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366643 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc"] Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366643 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.366701 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.366671 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q4dpp"] Jul 06 19:20:21.368247 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.368225 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.368341 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.368258 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.372886 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.372855 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 19:20:21.372983 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.372873 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.373043 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.372993 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-mgz4x\"" Jul 06 19:20:21.373105 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.373077 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 19:20:21.373433 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.373414 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.377981 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.377939 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.378677 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.378644 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjf2w\" (UniqueName: \"kubernetes.io/projected/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-kube-api-access-wjf2w\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.380179 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.380125 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rhl\" (UniqueName: \"kubernetes.io/projected/cd0e201a-7fcf-41b4-a2e3-49fae83fba8d-kube-api-access-c9rhl\") pod \"volume-data-source-validator-5d456fdbcd-2hrr8\" (UID: \"cd0e201a-7fcf-41b4-a2e3-49fae83fba8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" Jul 06 19:20:21.380471 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.380451 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cbm\" (UniqueName: \"kubernetes.io/projected/9ae5131a-c00c-4876-912f-bba5f0b6281e-kube-api-access-94cbm\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.380534 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.380520 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gdb7\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.391009 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.390990 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5"] Jul 06 19:20:21.391641 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.391621 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.396069 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.396050 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.396171 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.396105 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jkg7b\"" Jul 06 19:20:21.396171 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.396106 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:20:21.396804 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.396778 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.413231 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.413211 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx"] Jul 06 19:20:21.413323 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.413250 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-696bd7f7cc-8fbvx"] Jul 06 19:20:21.413323 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.413275 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n97st"] Jul 06 19:20:21.413422 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.413367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" Jul 06 19:20:21.416836 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.416819 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:20:21.417601 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.417440 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:20:21.417601 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.417440 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mrbbj\"" Jul 06 19:20:21.435391 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.435375 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pl7cl"] Jul 06 19:20:21.435494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.435401 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4dpp"] Jul 06 19:20:21.435494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.435418 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n97st"] Jul 06 19:20:21.435494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.435439 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5"] Jul 06 19:20:21.435494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.435492 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.438397 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.438378 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:20:21.438513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.438457 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r5ggw\"" Jul 06 19:20:21.438575 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.438457 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:20:21.465302 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.465400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kxjj\" (UniqueName: \"kubernetes.io/projected/d7679d0b-cbec-44fd-b860-7c9337800263-kube-api-access-4kxjj\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.465400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.465400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4vnh\" (UniqueName: \"kubernetes.io/projected/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-kube-api-access-c4vnh\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.465561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4e96682-a1fb-4d21-b66e-f82a843dc66c-cert\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.465561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-config\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.465561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmrvr\" (UniqueName: \"kubernetes.io/projected/b4e96682-a1fb-4d21-b66e-f82a843dc66c-kube-api-access-vmrvr\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.465561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-config\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-default-certificate\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrw8w\" (UniqueName: \"kubernetes.io/projected/0763d32a-7979-4dcb-8fff-d5842fa0ee83-kube-api-access-vrw8w\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-snapshots\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.465756 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.465739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7679d0b-cbec-44fd-b860-7c9337800263-config-volume\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.466084 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.465859 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:21.965744609 +0000 UTC m=+36.645778386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:21.466210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466184 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.466283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466244 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-config\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.466357 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466334 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.466542 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/408aa770-e6eb-4eb9-b635-8e65281f4183-serving-cert\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.466542 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7679d0b-cbec-44fd-b860-7c9337800263-tmp-dir\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.466542 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnxrn\" (UniqueName: \"kubernetes.io/projected/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-kube-api-access-vnxrn\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.466542 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.466542 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-config\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f44fb294-1867-4650-a8c8-1a7d7cf0378b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466579 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-tmp\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.466608 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.466660 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:21.966636914 +0000 UTC m=+36.646670706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c7cc\" (UniqueName: \"kubernetes.io/projected/f44fb294-1867-4650-a8c8-1a7d7cf0378b-kube-api-access-9c7cc\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466716 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-stats-auth\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466745 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f44fb294-1867-4650-a8c8-1a7d7cf0378b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466785 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6brz8\" (UniqueName: \"kubernetes.io/projected/325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4-kube-api-access-6brz8\") pod \"network-check-source-989f8b8dc-vf6t5\" (UID: \"325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" Jul 06 19:20:21.466822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg9m8\" (UniqueName: \"kubernetes.io/projected/408aa770-e6eb-4eb9-b635-8e65281f4183-kube-api-access-zg9m8\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.467295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466853 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-serving-cert\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.467295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466878 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-trusted-ca\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.467295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.466887 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-tmp\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.467295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.467000 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-snapshots\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.467775 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.467729 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/408aa770-e6eb-4eb9-b635-8e65281f4183-trusted-ca\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.468386 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.468323 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.468831 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.468809 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-default-certificate\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.469589 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.469528 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-stats-auth\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.469696 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.469632 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/408aa770-e6eb-4eb9-b635-8e65281f4183-serving-cert\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.470591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.470574 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-serving-cert\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.478979 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.478958 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrw8w\" (UniqueName: \"kubernetes.io/projected/0763d32a-7979-4dcb-8fff-d5842fa0ee83-kube-api-access-vrw8w\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.480447 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.480425 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4vnh\" (UniqueName: \"kubernetes.io/projected/134871dc-e7c2-4a57-a4c4-2fa19ddf5aed-kube-api-access-c4vnh\") pod \"service-ca-operator-5d46d9ffd4-gqdvx\" (UID: \"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.484478 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.484452 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg9m8\" (UniqueName: \"kubernetes.io/projected/408aa770-e6eb-4eb9-b635-8e65281f4183-kube-api-access-zg9m8\") pod \"console-operator-68b757fb67-pl7cl\" (UID: \"408aa770-e6eb-4eb9-b635-8e65281f4183\") " pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.485481 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.485462 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnxrn\" (UniqueName: \"kubernetes.io/projected/9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65-kube-api-access-vnxrn\") pod \"insights-operator-5c847dcfc4-gdrrs\" (UID: \"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65\") " pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.548557 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.548527 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" Jul 06 19:20:21.567955 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.567924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4e96682-a1fb-4d21-b66e-f82a843dc66c-cert\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.568112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568000 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmrvr\" (UniqueName: \"kubernetes.io/projected/b4e96682-a1fb-4d21-b66e-f82a843dc66c-kube-api-access-vmrvr\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.568112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568070 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7679d0b-cbec-44fd-b860-7c9337800263-config-volume\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568094 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7679d0b-cbec-44fd-b860-7c9337800263-tmp-dir\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568351 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.568192 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:21.568351 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.568252 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.068231938 +0000 UTC m=+36.748265717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:21.568462 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f44fb294-1867-4650-a8c8-1a7d7cf0378b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.568462 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c7cc\" (UniqueName: \"kubernetes.io/projected/f44fb294-1867-4650-a8c8-1a7d7cf0378b-kube-api-access-9c7cc\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.568462 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568424 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7679d0b-cbec-44fd-b860-7c9337800263-tmp-dir\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568462 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568445 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f44fb294-1867-4650-a8c8-1a7d7cf0378b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.568662 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568497 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6brz8\" (UniqueName: \"kubernetes.io/projected/325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4-kube-api-access-6brz8\") pod \"network-check-source-989f8b8dc-vf6t5\" (UID: \"325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" Jul 06 19:20:21.568662 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kxjj\" (UniqueName: \"kubernetes.io/projected/d7679d0b-cbec-44fd-b860-7c9337800263-kube-api-access-4kxjj\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568662 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568631 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d7679d0b-cbec-44fd-b860-7c9337800263-config-volume\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.568924 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.568903 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f44fb294-1867-4650-a8c8-1a7d7cf0378b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.570769 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.570746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f44fb294-1867-4650-a8c8-1a7d7cf0378b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.570884 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.570856 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4e96682-a1fb-4d21-b66e-f82a843dc66c-cert\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.581613 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.581588 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c7cc\" (UniqueName: \"kubernetes.io/projected/f44fb294-1867-4650-a8c8-1a7d7cf0378b-kube-api-access-9c7cc\") pod \"kube-storage-version-migrator-operator-846bf668f7-wm6gc\" (UID: \"f44fb294-1867-4650-a8c8-1a7d7cf0378b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.583374 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.583333 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmrvr\" (UniqueName: \"kubernetes.io/projected/b4e96682-a1fb-4d21-b66e-f82a843dc66c-kube-api-access-vmrvr\") pod \"ingress-canary-q4dpp\" (UID: \"b4e96682-a1fb-4d21-b66e-f82a843dc66c\") " pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.583910 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.583890 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6brz8\" (UniqueName: \"kubernetes.io/projected/325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4-kube-api-access-6brz8\") pod \"network-check-source-989f8b8dc-vf6t5\" (UID: \"325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" Jul 06 19:20:21.583910 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.583901 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kxjj\" (UniqueName: \"kubernetes.io/projected/d7679d0b-cbec-44fd-b860-7c9337800263-kube-api-access-4kxjj\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:21.597815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.597790 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" Jul 06 19:20:21.616233 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.616174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:21.662292 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.662117 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" Jul 06 19:20:21.700314 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.699301 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" Jul 06 19:20:21.704591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.704567 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q4dpp" Jul 06 19:20:21.724261 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.724198 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" Jul 06 19:20:21.816626 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.816335 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8"] Jul 06 19:20:21.842249 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:21.837676 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0e201a_7fcf_41b4_a2e3_49fae83fba8d.slice/crio-0c4471bd3ee30e1a2ce182ec25b0aa92e70833668c808da7d8d73292df7dadb6 WatchSource:0}: Error finding container 0c4471bd3ee30e1a2ce182ec25b0aa92e70833668c808da7d8d73292df7dadb6: Status 404 returned error can't find the container with id 0c4471bd3ee30e1a2ce182ec25b0aa92e70833668c808da7d8d73292df7dadb6 Jul 06 19:20:21.842661 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.842349 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-gdrrs"] Jul 06 19:20:21.843462 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.843397 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:21.844882 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.843689 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:21.848722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.848067 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r9p6l\"" Jul 06 19:20:21.848722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.848402 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c88qg\"" Jul 06 19:20:21.848722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.848582 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:20:21.850470 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.850273 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pl7cl"] Jul 06 19:20:21.872803 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.872779 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:21.872926 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.872891 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:21.872996 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.872925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:21.873112 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873029 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:21.873112 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873077 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.873058875 +0000 UTC m=+37.553092652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:21.873458 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873158 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:21.873458 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873194 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.873181441 +0000 UTC m=+37.553215219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:21.873458 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873257 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:21.873458 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873268 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:21.873458 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.873300 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.873289969 +0000 UTC m=+37.553323744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:21.919610 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.918522 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx"] Jul 06 19:20:21.923181 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:21.923150 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134871dc_e7c2_4a57_a4c4_2fa19ddf5aed.slice/crio-41069c39624624624d0b24c35b878e7746c0e3f2ce963a0c15fe67c5b4bb4802 WatchSource:0}: Error finding container 41069c39624624624d0b24c35b878e7746c0e3f2ce963a0c15fe67c5b4bb4802: Status 404 returned error can't find the container with id 41069c39624624624d0b24c35b878e7746c0e3f2ce963a0c15fe67c5b4bb4802 Jul 06 19:20:21.967369 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.967341 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc"] Jul 06 19:20:21.970536 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:21.970512 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44fb294_1867_4650_a8c8_1a7d7cf0378b.slice/crio-6b69401cf3d5ddb49fc98899cf1596c9b384272913e1f3a2f83546652694d7a5 WatchSource:0}: Error finding container 6b69401cf3d5ddb49fc98899cf1596c9b384272913e1f3a2f83546652694d7a5: Status 404 returned error can't find the container with id 6b69401cf3d5ddb49fc98899cf1596c9b384272913e1f3a2f83546652694d7a5 Jul 06 19:20:21.973585 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.973430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.973585 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.973515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:21.973703 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.973624 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.973603864 +0000 UTC m=+37.653637663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:21.973703 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.973688 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:21.973772 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:21.973743 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:22.973723489 +0000 UTC m=+37.653757264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:21.977083 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.977062 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q4dpp"] Jul 06 19:20:21.980099 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.980076 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5"] Jul 06 19:20:21.986777 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.986755 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" event={"ID":"cd0e201a-7fcf-41b4-a2e3-49fae83fba8d","Type":"ContainerStarted","Data":"0c4471bd3ee30e1a2ce182ec25b0aa92e70833668c808da7d8d73292df7dadb6"} Jul 06 19:20:21.987980 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.987949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" event={"ID":"f44fb294-1867-4650-a8c8-1a7d7cf0378b","Type":"ContainerStarted","Data":"6b69401cf3d5ddb49fc98899cf1596c9b384272913e1f3a2f83546652694d7a5"} Jul 06 19:20:21.989185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.989160 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" event={"ID":"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed","Type":"ContainerStarted","Data":"41069c39624624624d0b24c35b878e7746c0e3f2ce963a0c15fe67c5b4bb4802"} Jul 06 19:20:21.990685 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.990664 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" event={"ID":"408aa770-e6eb-4eb9-b635-8e65281f4183","Type":"ContainerStarted","Data":"86b2787a105624b6d2883def978b1b1eea940ef8aa6bc3fd30104a760e4c2a1a"} Jul 06 19:20:21.991796 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:21.991771 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e96682_a1fb_4d21_b66e_f82a843dc66c.slice/crio-012169242ae8285c4a5c325750ad411e4c0ff58aab5272b1eade614ee91ca88f WatchSource:0}: Error finding container 012169242ae8285c4a5c325750ad411e4c0ff58aab5272b1eade614ee91ca88f: Status 404 returned error can't find the container with id 012169242ae8285c4a5c325750ad411e4c0ff58aab5272b1eade614ee91ca88f Jul 06 19:20:21.992025 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:21.992005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" event={"ID":"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65","Type":"ContainerStarted","Data":"272c5f1f960c876c77e748faec7daf2702376f0dd354b92564b93753487cc459"} Jul 06 19:20:21.992588 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:21.992564 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325fa4ad_74a4_45eb_93d1_f87d0fa3c3b4.slice/crio-bda6ba9dd305bac5b2e7b91d2442677ca10070c3875b4eb7f125273bcd879a9b WatchSource:0}: Error finding container bda6ba9dd305bac5b2e7b91d2442677ca10070c3875b4eb7f125273bcd879a9b: Status 404 returned error can't find the container with id bda6ba9dd305bac5b2e7b91d2442677ca10070c3875b4eb7f125273bcd879a9b Jul 06 19:20:22.074785 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.074753 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:22.074931 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.074915 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:22.075006 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.074984 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:23.074964965 +0000 UTC m=+37.754998744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:22.843342 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.843306 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:22.846739 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.846716 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:20:22.883250 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.883220 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:22.883411 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.883392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:22.883495 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.883434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:22.883638 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.883622 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:22.883705 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.883689 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:24.883670783 +0000 UTC m=+39.563704561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:22.884116 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.884094 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:22.884227 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.884165 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:24.884149644 +0000 UTC m=+39.564183419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:22.884304 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.884237 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:22.884304 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.884249 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:22.884304 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.884298 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:24.884286026 +0000 UTC m=+39.564319803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:22.984591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.984556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:22.984752 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:22.984636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:22.984752 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.984734 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:24.984713104 +0000 UTC m=+39.664746884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:22.984869 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.984797 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:22.984869 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:22.984847 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:24.984830957 +0000 UTC m=+39.664864733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:23.008705 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:23.008668 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" event={"ID":"325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4","Type":"ContainerStarted","Data":"bda6ba9dd305bac5b2e7b91d2442677ca10070c3875b4eb7f125273bcd879a9b"} Jul 06 19:20:23.011814 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:23.011768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4dpp" event={"ID":"b4e96682-a1fb-4d21-b66e-f82a843dc66c","Type":"ContainerStarted","Data":"012169242ae8285c4a5c325750ad411e4c0ff58aab5272b1eade614ee91ca88f"} Jul 06 19:20:23.086283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:23.086242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:23.086527 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:23.086508 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:23.086601 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:23.086579 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:25.086559227 +0000 UTC m=+39.766593003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:24.902963 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:24.902926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:24.903009 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:24.903028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903109 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903157 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903202 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:28.903183627 +0000 UTC m=+43.583217424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903220 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:28.903212877 +0000 UTC m=+43.583246652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903159 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903231 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:24.903437 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:24.903259 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:28.903250336 +0000 UTC m=+43.583284111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:25.004550 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:25.004516 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:25.004715 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:25.004576 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:25.004715 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:25.004685 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:25.004818 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:25.004688 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:29.004670977 +0000 UTC m=+43.684704756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:25.004818 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:25.004749 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:29.004733878 +0000 UTC m=+43.684767666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:25.105356 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:25.105306 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:25.105583 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:25.105487 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:25.105583 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:25.105561 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:29.105542319 +0000 UTC m=+43.785576097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:28.942330 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:28.942293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:28.942471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:28.942501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942504 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942579 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:36.942560557 +0000 UTC m=+51.622594340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942618 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942628 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942644 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942670 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:36.942658764 +0000 UTC m=+51.622692539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:28.942733 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:28.942689 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:36.94267838 +0000 UTC m=+51.622712178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:29.043683 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:29.043658 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:29.043851 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:29.043701 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:29.043851 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:29.043794 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:29.043851 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:29.043827 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:37.043809624 +0000 UTC m=+51.723843423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:29.043851 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:29.043851 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:37.04384169 +0000 UTC m=+51.723875469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:29.144816 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:29.144787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:29.144931 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:29.144897 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:29.144982 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:29.144938 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:37.14492593 +0000 UTC m=+51.824959705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:32.039239 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.038891 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="34210d22ed498f3537c89b64757bdedd68ac4ab59115b662eadec1208a8b2128" exitCode=0 Jul 06 19:20:32.039984 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.038939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"34210d22ed498f3537c89b64757bdedd68ac4ab59115b662eadec1208a8b2128"} Jul 06 19:20:32.042910 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.042856 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q4dpp" event={"ID":"b4e96682-a1fb-4d21-b66e-f82a843dc66c","Type":"ContainerStarted","Data":"c16838d9dce3183fe5bb3d39fe2b6033cb7740cd19c27ce1f978f42c493cdf5d"} Jul 06 19:20:32.044781 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.044752 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" event={"ID":"cd0e201a-7fcf-41b4-a2e3-49fae83fba8d","Type":"ContainerStarted","Data":"b3fb6fc0ac1631a5cb37bdea3860d58f43f8fa52f0f50f44c327f5c587ceb9e7"} Jul 06 19:20:32.047061 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.046765 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" event={"ID":"f44fb294-1867-4650-a8c8-1a7d7cf0378b","Type":"ContainerStarted","Data":"5604692838fd242ddd3922a1c8eca2d12e2656d2c75facbd876166992eaa180c"} Jul 06 19:20:32.048457 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.048435 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" event={"ID":"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed","Type":"ContainerStarted","Data":"79c9fd16308e2793ab62edeb6e70af49d8e2a5a151b3ecb19d9115745ead189f"} Jul 06 19:20:32.050335 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.050315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/0.log" Jul 06 19:20:32.050435 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.050354 2576 generic.go:356] "Generic (PLEG): container finished" podID="408aa770-e6eb-4eb9-b635-8e65281f4183" containerID="0d4eb39d552fd50c0328ce3d4e39402853d22a78a8e4a8fd8418b33f5c220f79" exitCode=255 Jul 06 19:20:32.050435 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.050426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" event={"ID":"408aa770-e6eb-4eb9-b635-8e65281f4183","Type":"ContainerDied","Data":"0d4eb39d552fd50c0328ce3d4e39402853d22a78a8e4a8fd8418b33f5c220f79"} Jul 06 19:20:32.050676 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.050638 2576 scope.go:117] "RemoveContainer" containerID="0d4eb39d552fd50c0328ce3d4e39402853d22a78a8e4a8fd8418b33f5c220f79" Jul 06 19:20:32.052182 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.052111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" event={"ID":"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65","Type":"ContainerStarted","Data":"c0db11153d8245caf9362f694996770164f5c001e3104b652bea9cf4d1ec2020"} Jul 06 19:20:32.054159 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.054114 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" event={"ID":"325fa4ad-74a4-45eb-93d1-f87d0fa3c3b4","Type":"ContainerStarted","Data":"db0ca59f8b9a6b4a671a70dee5c3ae9120ee5306bab599380732679959fb55a2"} Jul 06 19:20:32.085896 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.085838 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" podStartSLOduration=12.857779532 podStartE2EDuration="22.085819371s" totalImagesPullingTime="9.228039839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.084643265 +0000 UTC m=+46.764677062" watchObservedRunningTime="2026-07-06 19:20:32.085819371 +0000 UTC m=+46.765853170" Jul 06 19:20:32.205650 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.205200 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q4dpp" podStartSLOduration=2.098138275 podStartE2EDuration="11.205179684s" totalImagesPullingTime="9.107041409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.20459459 +0000 UTC m=+46.884628390" watchObservedRunningTime="2026-07-06 19:20:32.205179684 +0000 UTC m=+46.885213475" Jul 06 19:20:32.205650 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.205452 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" podStartSLOduration=13.077792364 podStartE2EDuration="22.205443917s" totalImagesPullingTime="9.127651553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.155344633 +0000 UTC m=+46.835378431" watchObservedRunningTime="2026-07-06 19:20:32.205443917 +0000 UTC m=+46.885477718" Jul 06 19:20:32.267056 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.267001 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-vf6t5" podStartSLOduration=13.149875411 podStartE2EDuration="22.266980819s" totalImagesPullingTime="9.117105408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.264411942 +0000 UTC m=+46.944445740" watchObservedRunningTime="2026-07-06 19:20:32.266980819 +0000 UTC m=+46.947014617" Jul 06 19:20:32.287945 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.287860 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2hrr8" podStartSLOduration=14.358752435 podStartE2EDuration="23.28784027s" totalImagesPullingTime="8.929087835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.28750822 +0000 UTC m=+46.967542018" watchObservedRunningTime="2026-07-06 19:20:32.28784027 +0000 UTC m=+46.967874067" Jul 06 19:20:32.312189 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:32.311027 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" podStartSLOduration=13.136628631 podStartE2EDuration="22.311006588s" totalImagesPullingTime="9.174377957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:32.307298489 +0000 UTC m=+46.987332288" watchObservedRunningTime="2026-07-06 19:20:32.311006588 +0000 UTC m=+46.991040386" Jul 06 19:20:33.060015 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.059982 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:20:33.060552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.060532 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/0.log" Jul 06 19:20:33.060654 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.060577 2576 generic.go:356] "Generic (PLEG): container finished" podID="408aa770-e6eb-4eb9-b635-8e65281f4183" containerID="6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688" exitCode=255 Jul 06 19:20:33.060709 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.060686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" event={"ID":"408aa770-e6eb-4eb9-b635-8e65281f4183","Type":"ContainerDied","Data":"6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688"} Jul 06 19:20:33.060809 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.060787 2576 scope.go:117] "RemoveContainer" containerID="0d4eb39d552fd50c0328ce3d4e39402853d22a78a8e4a8fd8418b33f5c220f79" Jul 06 19:20:33.061064 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.061049 2576 scope.go:117] "RemoveContainer" containerID="6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688" Jul 06 19:20:33.061481 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.061269 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pl7cl_openshift-console-operator(408aa770-e6eb-4eb9-b635-8e65281f4183)\"" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" podUID="408aa770-e6eb-4eb9-b635-8e65281f4183" Jul 06 19:20:33.064591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.064571 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9094bf-9ca0-459f-9b2b-9f3816c2eb89" containerID="125eba66c407b2d660b0d061cad02b2e6a707e432bce59cf9c60ea1e1f50cd9d" exitCode=0 Jul 06 19:20:33.064676 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.064660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerDied","Data":"125eba66c407b2d660b0d061cad02b2e6a707e432bce59cf9c60ea1e1f50cd9d"} Jul 06 19:20:33.329174 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.329072 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7nz4z"] Jul 06 19:20:33.346250 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.346214 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7nz4z"] Jul 06 19:20:33.346384 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.346334 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.349929 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.349908 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:20:33.349929 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.349924 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:20:33.350306 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.350292 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2lnlz\"" Jul 06 19:20:33.384529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.384499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-data-volume\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.384663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.384552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.384663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.384602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.384663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.384627 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d7b\" (UniqueName: \"kubernetes.io/projected/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-api-access-j2d7b\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.384813 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.384801 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-crio-socket\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.485906 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.485874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-crio-socket\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486065 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.485938 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-data-volume\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486065 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.485997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486065 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.486057 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486258 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.486091 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d7b\" (UniqueName: \"kubernetes.io/projected/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-api-access-j2d7b\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486258 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.486101 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:20:33.486258 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.486126 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-crio-socket\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.486258 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.486191 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:33.986172958 +0000 UTC m=+48.666206734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : configmap "kube-rbac-proxy" not found Jul 06 19:20:33.486258 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.486239 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:33.486495 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.486299 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:33.986283144 +0000 UTC m=+48.666316934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:33.486495 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.486284 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-data-volume\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.500164 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.500116 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d7b\" (UniqueName: \"kubernetes.io/projected/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-api-access-j2d7b\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.989190 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.989158 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.989347 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:33.989198 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:33.989347 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.989269 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:20:33.989347 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.989336 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:34.989319777 +0000 UTC m=+49.669353553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : configmap "kube-rbac-proxy" not found Jul 06 19:20:33.989505 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.989379 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:33.989505 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:33.989445 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:34.989427549 +0000 UTC m=+49.669461328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:34.074725 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.074701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:20:34.075113 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.075020 2576 scope.go:117] "RemoveContainer" containerID="6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688" Jul 06 19:20:34.075225 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:34.075200 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pl7cl_openshift-console-operator(408aa770-e6eb-4eb9-b635-8e65281f4183)\"" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" podUID="408aa770-e6eb-4eb9-b635-8e65281f4183" Jul 06 19:20:34.077983 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.077959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" event={"ID":"1b9094bf-9ca0-459f-9b2b-9f3816c2eb89","Type":"ContainerStarted","Data":"ffd1b43b4e9b850937fe50307f437c3939724306545a19c8ee197d63d7107f5c"} Jul 06 19:20:34.114949 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.114906 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t2vpz" podStartSLOduration=13.334586985 podStartE2EDuration="49.114892204s" totalImagesPullingTime="35.780305219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:34.11384961 +0000 UTC m=+48.793883409" watchObservedRunningTime="2026-07-06 19:20:34.114892204 +0000 UTC m=+48.794925995" Jul 06 19:20:34.997590 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.997554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:34.997739 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:34.997598 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:34.997739 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:34.997676 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:20:34.997739 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:34.997736 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:36.997721975 +0000 UTC m=+51.677755753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : configmap "kube-rbac-proxy" not found Jul 06 19:20:34.997862 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:34.997780 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:34.997862 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:34.997843 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:36.997829689 +0000 UTC m=+51.677863464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:35.930144 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:35.930116 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvh5p_cf9bedb2-db52-4c8a-96f3-2834416a1c76/dns-node-resolver/0.log" Jul 06 19:20:36.530607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:36.530581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5rc7_b7e4997e-01f4-4909-a2a6-3eee839cf20b/node-ca/0.log" Jul 06 19:20:37.016890 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.016856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017010 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.017028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017083 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls podName:9ae5131a-c00c-4876-912f-bba5f0b6281e nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.017064871 +0000 UTC m=+67.697098669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-h5pwl" (UID: "9ae5131a-c00c-4876-912f-bba5f0b6281e") : secret "samples-operator-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017108 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017168 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:41.017155908 +0000 UTC m=+55.697189688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : configmap "kube-rbac-proxy" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.017105 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017191 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017225 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:41.017214785 +0000 UTC m=+55.697248562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.017241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.017265 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017347 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017356 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:37.017366 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017359 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d8d6779cf-rgklz: secret "image-registry-tls" not found Jul 06 19:20:37.017818 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017394 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls podName:3581fad1-94cd-42ee-9aa7-abb78fa9cfec nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.017382644 +0000 UTC m=+67.697416445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4z89j" (UID: "3581fad1-94cd-42ee-9aa7-abb78fa9cfec") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:20:37.017818 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.017407 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls podName:b8d93d2c-cf57-4578-9195-916e6987a547 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.017401583 +0000 UTC m=+67.697435358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls") pod "image-registry-d8d6779cf-rgklz" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547") : secret "image-registry-tls" not found Jul 06 19:20:37.118248 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.118217 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:37.118412 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.118286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:37.118412 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.118379 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 19:20:37.118412 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.118383 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.118364279 +0000 UTC m=+67.798398054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : configmap references non-existent config key: service-ca.crt Jul 06 19:20:37.118541 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.118431 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs podName:0763d32a-7979-4dcb-8fff-d5842fa0ee83 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.118418844 +0000 UTC m=+67.798452633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs") pod "router-default-696bd7f7cc-8fbvx" (UID: "0763d32a-7979-4dcb-8fff-d5842fa0ee83") : secret "router-metrics-certs-default" not found Jul 06 19:20:37.219279 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.219243 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:37.219462 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.219388 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:20:37.219462 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:37.219451 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls podName:d7679d0b-cbec-44fd-b860-7c9337800263 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:53.219433246 +0000 UTC m=+67.899467038 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls") pod "dns-default-n97st" (UID: "d7679d0b-cbec-44fd-b860-7c9337800263") : secret "dns-default-metrics-tls" not found Jul 06 19:20:37.332546 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:37.332482 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q4dpp_b4e96682-a1fb-4d21-b66e-f82a843dc66c/serve-healthcheck-canary/0.log" Jul 06 19:20:38.130823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:38.130797 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wm6gc_f44fb294-1867-4650-a8c8-1a7d7cf0378b/kube-storage-version-migrator-operator/0.log" Jul 06 19:20:41.051993 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:41.051955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:41.051993 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:41.051998 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:41.052511 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:41.052079 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:20:41.052511 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:41.052099 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:41.052511 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:41.052184 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:49.052167933 +0000 UTC m=+63.732201711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:41.052511 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:41.052226 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:20:49.052218058 +0000 UTC m=+63.732251833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : configmap "kube-rbac-proxy" not found Jul 06 19:20:41.616593 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:41.616545 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:41.616771 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:41.616614 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:41.616942 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:41.616931 2576 scope.go:117] "RemoveContainer" containerID="6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688" Jul 06 19:20:41.617149 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:41.617083 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pl7cl_openshift-console-operator(408aa770-e6eb-4eb9-b635-8e65281f4183)\"" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" podUID="408aa770-e6eb-4eb9-b635-8e65281f4183" Jul 06 19:20:45.987420 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:45.987395 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pl2cw" Jul 06 19:20:49.117643 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:49.117605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:49.118054 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:49.117657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:49.118054 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:49.117819 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:20:49.118054 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:49.117885 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls podName:0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26 nodeName:}" failed. No retries permitted until 2026-07-06 19:21:05.117865997 +0000 UTC m=+79.797899772 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nz4z" (UID: "0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26") : secret "insights-runtime-extractor-tls" not found Jul 06 19:20:49.118195 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:49.118082 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:20:51.637316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.637282 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:51.637316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.637318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:51.637792 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.637554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:51.640005 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.639986 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8s2v\" (UniqueName: \"kubernetes.io/projected/66da0fd3-366a-45a3-b755-7fcc37389e00-kube-api-access-n8s2v\") pod \"network-check-target-s2wf7\" (UID: \"66da0fd3-366a-45a3-b755-7fcc37389e00\") " pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:51.643837 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.643813 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:20:51.644712 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.644695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:20:51.649669 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.649646 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e69ee40d-db1c-4469-929c-9338f79a9e97-original-pull-secret\") pod \"global-pull-secret-syncer-lr9nc\" (UID: \"e69ee40d-db1c-4469-929c-9338f79a9e97\") " pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:51.649761 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.649748 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24b96433-844c-41a0-9cef-dacde28c440f-metrics-certs\") pod \"network-metrics-daemon-68mm4\" (UID: \"24b96433-844c-41a0-9cef-dacde28c440f\") " pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:51.657430 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.657411 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lr9nc" Jul 06 19:20:51.778044 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.777999 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lr9nc"] Jul 06 19:20:51.780986 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:51.780963 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69ee40d_db1c_4469_929c_9338f79a9e97.slice/crio-63c8fc3e592c333afeb2f0625833d63168c2ecbd83bbb2eb6b887a1ae8d67032 WatchSource:0}: Error finding container 63c8fc3e592c333afeb2f0625833d63168c2ecbd83bbb2eb6b887a1ae8d67032: Status 404 returned error can't find the container with id 63c8fc3e592c333afeb2f0625833d63168c2ecbd83bbb2eb6b887a1ae8d67032 Jul 06 19:20:51.892169 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.892091 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r9p6l\"" Jul 06 19:20:51.896697 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.896682 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c88qg\"" Jul 06 19:20:51.899778 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.899765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:51.904847 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:51.904833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-68mm4" Jul 06 19:20:52.026060 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.026024 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2wf7"] Jul 06 19:20:52.029458 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:52.029429 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66da0fd3_366a_45a3_b755_7fcc37389e00.slice/crio-354596c0fc41c37417d8059bc1737b3cfcc6ea0f501417b96ddc010c66354157 WatchSource:0}: Error finding container 354596c0fc41c37417d8059bc1737b3cfcc6ea0f501417b96ddc010c66354157: Status 404 returned error can't find the container with id 354596c0fc41c37417d8059bc1737b3cfcc6ea0f501417b96ddc010c66354157 Jul 06 19:20:52.041021 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.041001 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-68mm4"] Jul 06 19:20:52.043024 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:52.043000 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b96433_844c_41a0_9cef_dacde28c440f.slice/crio-299fce1b9b02c4bfcac7c7d2f667d5ea2e344ac224ccdddb606ad20ce870fb32 WatchSource:0}: Error finding container 299fce1b9b02c4bfcac7c7d2f667d5ea2e344ac224ccdddb606ad20ce870fb32: Status 404 returned error can't find the container with id 299fce1b9b02c4bfcac7c7d2f667d5ea2e344ac224ccdddb606ad20ce870fb32 Jul 06 19:20:52.134430 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.134398 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-68mm4" event={"ID":"24b96433-844c-41a0-9cef-dacde28c440f","Type":"ContainerStarted","Data":"299fce1b9b02c4bfcac7c7d2f667d5ea2e344ac224ccdddb606ad20ce870fb32"} Jul 06 19:20:52.135849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.135827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2wf7" event={"ID":"66da0fd3-366a-45a3-b755-7fcc37389e00","Type":"ContainerStarted","Data":"1a68bfe701a8f22f9713fa11d48466b25368078d30c55019bed546ae0a172132"} Jul 06 19:20:52.135951 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.135863 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2wf7" event={"ID":"66da0fd3-366a-45a3-b755-7fcc37389e00","Type":"ContainerStarted","Data":"354596c0fc41c37417d8059bc1737b3cfcc6ea0f501417b96ddc010c66354157"} Jul 06 19:20:52.135951 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.135897 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:20:52.136855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.136830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lr9nc" event={"ID":"e69ee40d-db1c-4469-929c-9338f79a9e97","Type":"ContainerStarted","Data":"63c8fc3e592c333afeb2f0625833d63168c2ecbd83bbb2eb6b887a1ae8d67032"} Jul 06 19:20:52.153911 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:52.153822 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s2wf7" podStartSLOduration=66.153807678 podStartE2EDuration="1m6.153807678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:52.153586215 +0000 UTC m=+66.833620013" watchObservedRunningTime="2026-07-06 19:20:52.153807678 +0000 UTC m=+66.833841472" Jul 06 19:20:53.049373 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.049341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:53.049373 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.049379 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:53.049893 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.049426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:53.052191 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.052164 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ae5131a-c00c-4876-912f-bba5f0b6281e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-h5pwl\" (UID: \"9ae5131a-c00c-4876-912f-bba5f0b6281e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:53.052727 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.052696 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3581fad1-94cd-42ee-9aa7-abb78fa9cfec-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4z89j\" (UID: \"3581fad1-94cd-42ee-9aa7-abb78fa9cfec\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:53.052849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.052806 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"image-registry-d8d6779cf-rgklz\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:53.077110 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.077085 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rk7gb\"" Jul 06 19:20:53.085257 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.085236 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" Jul 06 19:20:53.150394 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.150366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:53.150558 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.150418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:53.151185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.151162 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0763d32a-7979-4dcb-8fff-d5842fa0ee83-service-ca-bundle\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:53.153093 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.153071 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0763d32a-7979-4dcb-8fff-d5842fa0ee83-metrics-certs\") pod \"router-default-696bd7f7cc-8fbvx\" (UID: \"0763d32a-7979-4dcb-8fff-d5842fa0ee83\") " pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:53.251504 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.251472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:53.254589 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.254531 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d7679d0b-cbec-44fd-b860-7c9337800263-metrics-tls\") pod \"dns-default-n97st\" (UID: \"d7679d0b-cbec-44fd-b860-7c9337800263\") " pod="openshift-dns/dns-default-n97st" Jul 06 19:20:53.295501 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.295476 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j"] Jul 06 19:20:53.298242 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:53.298219 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3581fad1_94cd_42ee_9aa7_abb78fa9cfec.slice/crio-aaa424f4381869a564819d0d605a93fd6fe24ade91bba773a4a5a82d3a0f760d WatchSource:0}: Error finding container aaa424f4381869a564819d0d605a93fd6fe24ade91bba773a4a5a82d3a0f760d: Status 404 returned error can't find the container with id aaa424f4381869a564819d0d605a93fd6fe24ade91bba773a4a5a82d3a0f760d Jul 06 19:20:53.322319 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.322296 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zrfbt\"" Jul 06 19:20:53.330563 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.330538 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:53.337740 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.337704 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-gzqb5\"" Jul 06 19:20:53.345522 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.345388 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" Jul 06 19:20:53.442930 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.442868 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qwzqp\"" Jul 06 19:20:53.451679 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.451652 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:53.482510 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.482441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:20:53.488958 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:53.488936 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d93d2c_cf57_4578_9195_916e6987a547.slice/crio-1d16f7bff4b1783e9f5608ee2b5701b4c861d4874de2512e42645b78fe3036dd WatchSource:0}: Error finding container 1d16f7bff4b1783e9f5608ee2b5701b4c861d4874de2512e42645b78fe3036dd: Status 404 returned error can't find the container with id 1d16f7bff4b1783e9f5608ee2b5701b4c861d4874de2512e42645b78fe3036dd Jul 06 19:20:53.503689 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.503618 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl"] Jul 06 19:20:53.547930 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.547907 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r5ggw\"" Jul 06 19:20:53.556310 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.556103 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n97st" Jul 06 19:20:53.610590 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.610565 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-696bd7f7cc-8fbvx"] Jul 06 19:20:53.615802 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:53.615755 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0763d32a_7979_4dcb_8fff_d5842fa0ee83.slice/crio-429ba81306a29d666b8eea816cb688991100087c916f0755d8d774380397f085 WatchSource:0}: Error finding container 429ba81306a29d666b8eea816cb688991100087c916f0755d8d774380397f085: Status 404 returned error can't find the container with id 429ba81306a29d666b8eea816cb688991100087c916f0755d8d774380397f085 Jul 06 19:20:53.705422 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:53.705377 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n97st"] Jul 06 19:20:53.710486 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:53.710447 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7679d0b_cbec_44fd_b860_7c9337800263.slice/crio-2f4135d54365df4da072344741c58dcdff7bbc27322adf5619e3cebd94ec6893 WatchSource:0}: Error finding container 2f4135d54365df4da072344741c58dcdff7bbc27322adf5619e3cebd94ec6893: Status 404 returned error can't find the container with id 2f4135d54365df4da072344741c58dcdff7bbc27322adf5619e3cebd94ec6893 Jul 06 19:20:54.146267 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.146229 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n97st" event={"ID":"d7679d0b-cbec-44fd-b860-7c9337800263","Type":"ContainerStarted","Data":"2f4135d54365df4da072344741c58dcdff7bbc27322adf5619e3cebd94ec6893"} Jul 06 19:20:54.147538 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.147504 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" event={"ID":"9ae5131a-c00c-4876-912f-bba5f0b6281e","Type":"ContainerStarted","Data":"52bc316fb2025d3d3872ddf6210835ac353b37632aa95ccfc6468e880eb4f6c9"} Jul 06 19:20:54.148835 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.148814 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" event={"ID":"b8d93d2c-cf57-4578-9195-916e6987a547","Type":"ContainerStarted","Data":"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6"} Jul 06 19:20:54.148953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.148847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" event={"ID":"b8d93d2c-cf57-4578-9195-916e6987a547","Type":"ContainerStarted","Data":"1d16f7bff4b1783e9f5608ee2b5701b4c861d4874de2512e42645b78fe3036dd"} Jul 06 19:20:54.148953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.148944 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:20:54.150489 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.150466 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" event={"ID":"0763d32a-7979-4dcb-8fff-d5842fa0ee83","Type":"ContainerStarted","Data":"034dc20c57ce6c1f67504f2b41c169af9ab3fe15a16041eb2d389ab8f3501826"} Jul 06 19:20:54.150590 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.150503 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" event={"ID":"0763d32a-7979-4dcb-8fff-d5842fa0ee83","Type":"ContainerStarted","Data":"429ba81306a29d666b8eea816cb688991100087c916f0755d8d774380397f085"} Jul 06 19:20:54.151510 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.151494 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" event={"ID":"3581fad1-94cd-42ee-9aa7-abb78fa9cfec","Type":"ContainerStarted","Data":"aaa424f4381869a564819d0d605a93fd6fe24ade91bba773a4a5a82d3a0f760d"} Jul 06 19:20:54.152975 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.152957 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-68mm4" event={"ID":"24b96433-844c-41a0-9cef-dacde28c440f","Type":"ContainerStarted","Data":"f157f6ed244c55d3ef1fcf31122a5d1fedaa94c9e9606dc85f0206f336c3404a"} Jul 06 19:20:54.153050 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.152989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-68mm4" event={"ID":"24b96433-844c-41a0-9cef-dacde28c440f","Type":"ContainerStarted","Data":"9c19e10a71460435b9862c91163a604eea3157c4a4e1535fca78e566483a21c6"} Jul 06 19:20:54.217325 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.217273 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" podStartSLOduration=68.217257978 podStartE2EDuration="1m8.217257978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:54.186163534 +0000 UTC m=+68.866197331" watchObservedRunningTime="2026-07-06 19:20:54.217257978 +0000 UTC m=+68.897291768" Jul 06 19:20:54.261929 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.261303 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" podStartSLOduration=45.261279842 podStartE2EDuration="45.261279842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:54.259654339 +0000 UTC m=+68.939688137" watchObservedRunningTime="2026-07-06 19:20:54.261279842 +0000 UTC m=+68.941313639" Jul 06 19:20:54.262063 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.261972 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-68mm4" podStartSLOduration=67.12697229 podStartE2EDuration="1m8.261961243s" totalImagesPullingTime="1.134988953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:54.220732864 +0000 UTC m=+68.900766661" watchObservedRunningTime="2026-07-06 19:20:54.261961243 +0000 UTC m=+68.941995040" Jul 06 19:20:54.452156 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.452108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:54.455320 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.455272 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:54.844793 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:54.844391 2576 scope.go:117] "RemoveContainer" containerID="6143d95feccc66b9276d618628668e0964e172bfdf0f291af01d2ad11d2b1688" Jul 06 19:20:55.160458 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.160390 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:20:55.160877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.160570 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" event={"ID":"408aa770-e6eb-4eb9-b635-8e65281f4183","Type":"ContainerStarted","Data":"9d029ba68655dba4cc9aebf50734747626c15bed9dee0c390692c50046f94d9c"} Jul 06 19:20:55.161015 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.160956 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:55.161634 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.161556 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:55.162797 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.162380 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-696bd7f7cc-8fbvx" Jul 06 19:20:55.208318 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.208002 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" podStartSLOduration=35.979523378 podStartE2EDuration="45.207980192s" totalImagesPullingTime="9.228456814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:55.182305398 +0000 UTC m=+69.862339199" watchObservedRunningTime="2026-07-06 19:20:55.207980192 +0000 UTC m=+69.888013991" Jul 06 19:20:55.851468 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:55.851442 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-pl7cl" Jul 06 19:20:58.569456 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.568568 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6"] Jul 06 19:20:58.594232 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.594195 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg"] Jul 06 19:20:58.594400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.594347 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.603316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.602969 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 19:20:58.603316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.603075 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-flgbp\"" Jul 06 19:20:58.603316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.603252 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 19:20:58.603316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.603287 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 19:20:58.604331 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.604247 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 19:20:58.623102 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.622472 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6"] Jul 06 19:20:58.623102 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.622585 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.627357 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.626075 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 19:20:58.630174 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.630124 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg"] Jul 06 19:20:58.703894 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.703847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhvpl\" (UniqueName: \"kubernetes.io/projected/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-kube-api-access-rhvpl\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.704068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.703965 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.804766 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.804584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.804766 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.804638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86b2b1d9-e13b-4515-84a6-243ec5129f4c-klusterlet-config\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.804766 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.804675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b2b1d9-e13b-4515-84a6-243ec5129f4c-tmp\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.804766 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.804706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhvpl\" (UniqueName: \"kubernetes.io/projected/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-kube-api-access-rhvpl\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.804766 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.804735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbzdk\" (UniqueName: \"kubernetes.io/projected/86b2b1d9-e13b-4515-84a6-243ec5129f4c-kube-api-access-lbzdk\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.810021 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.808543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.821579 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.821474 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhvpl\" (UniqueName: \"kubernetes.io/projected/a987b070-f4fe-4af1-8564-6f2ae0be7eaa-kube-api-access-rhvpl\") pod \"managed-serviceaccount-addon-agent-569f7f8f88-j9zp6\" (UID: \"a987b070-f4fe-4af1-8564-6f2ae0be7eaa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.905859 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.905819 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86b2b1d9-e13b-4515-84a6-243ec5129f4c-klusterlet-config\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.906039 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.905872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b2b1d9-e13b-4515-84a6-243ec5129f4c-tmp\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.906039 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.905911 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbzdk\" (UniqueName: \"kubernetes.io/projected/86b2b1d9-e13b-4515-84a6-243ec5129f4c-kube-api-access-lbzdk\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.906490 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.906440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b2b1d9-e13b-4515-84a6-243ec5129f4c-tmp\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.908398 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.908370 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86b2b1d9-e13b-4515-84a6-243ec5129f4c-klusterlet-config\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.918207 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.918185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" Jul 06 19:20:58.921716 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.921694 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbzdk\" (UniqueName: \"kubernetes.io/projected/86b2b1d9-e13b-4515-84a6-243ec5129f4c-kube-api-access-lbzdk\") pod \"klusterlet-addon-workmgr-f867765d5-gl6cg\" (UID: \"86b2b1d9-e13b-4515-84a6-243ec5129f4c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:58.942187 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:58.942167 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:20:59.116002 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.115971 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6"] Jul 06 19:20:59.119368 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:59.119340 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda987b070_f4fe_4af1_8564_6f2ae0be7eaa.slice/crio-8b945b8a5d006e5404de99842bc1e4307939118bf892f2df0ad42d32a0746244 WatchSource:0}: Error finding container 8b945b8a5d006e5404de99842bc1e4307939118bf892f2df0ad42d32a0746244: Status 404 returned error can't find the container with id 8b945b8a5d006e5404de99842bc1e4307939118bf892f2df0ad42d32a0746244 Jul 06 19:20:59.133507 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.133483 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg"] Jul 06 19:20:59.136234 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:20:59.136203 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b2b1d9_e13b_4515_84a6_243ec5129f4c.slice/crio-630e0ef4d9ad9c2fbb97ac8ca454eb821d500d59c3854f7be4a1efbe43af7325 WatchSource:0}: Error finding container 630e0ef4d9ad9c2fbb97ac8ca454eb821d500d59c3854f7be4a1efbe43af7325: Status 404 returned error can't find the container with id 630e0ef4d9ad9c2fbb97ac8ca454eb821d500d59c3854f7be4a1efbe43af7325 Jul 06 19:20:59.175837 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.175808 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44"] Jul 06 19:20:59.201705 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.201661 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lr9nc" podStartSLOduration=66.543865472 podStartE2EDuration="1m13.201645726s" totalImagesPullingTime="6.657780254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:59.200517912 +0000 UTC m=+73.880551710" watchObservedRunningTime="2026-07-06 19:20:59.201645726 +0000 UTC m=+73.881679563" Jul 06 19:20:59.207536 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207515 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" event={"ID":"86b2b1d9-e13b-4515-84a6-243ec5129f4c","Type":"ContainerStarted","Data":"630e0ef4d9ad9c2fbb97ac8ca454eb821d500d59c3854f7be4a1efbe43af7325"} Jul 06 19:20:59.207607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207550 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n97st" Jul 06 19:20:59.207607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207573 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44"] Jul 06 19:20:59.207607 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207601 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lr9nc" event={"ID":"e69ee40d-db1c-4469-929c-9338f79a9e97","Type":"ContainerStarted","Data":"b5357046b6424a411e531e424e5282a2a853b971fa6f2c7cb9da104a178389f1"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207619 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" event={"ID":"3581fad1-94cd-42ee-9aa7-abb78fa9cfec","Type":"ContainerStarted","Data":"ee4d144b559ad208357fcbcad172db2ea2ea141300d02554e66de8f7b19d59d1"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207634 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" event={"ID":"a987b070-f4fe-4af1-8564-6f2ae0be7eaa","Type":"ContainerStarted","Data":"8b945b8a5d006e5404de99842bc1e4307939118bf892f2df0ad42d32a0746244"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207651 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n97st" event={"ID":"d7679d0b-cbec-44fd-b860-7c9337800263","Type":"ContainerStarted","Data":"8e5a77d59b2d2ed9076df19d7b304651a81b197b0633742856a1d37d8190c27e"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n97st" event={"ID":"d7679d0b-cbec-44fd-b860-7c9337800263","Type":"ContainerStarted","Data":"47b3bcad61b438f0779e40052b04879d31bf8366cca4bb375f05a20bd0fa1933"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207678 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" event={"ID":"9ae5131a-c00c-4876-912f-bba5f0b6281e","Type":"ContainerStarted","Data":"5117d5a91abbfd0dbc8b429890dc4c1b32146e40dc59edaab14e5fa6d4c61fe0"} Jul 06 19:20:59.207713 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207691 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:20:59.207893 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.207694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" event={"ID":"9ae5131a-c00c-4876-912f-bba5f0b6281e","Type":"ContainerStarted","Data":"06d0fe23be2038e015990b5db1d4b708dd541a3cd841e74f0c0e2ac92d85354f"} Jul 06 19:20:59.210092 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.210072 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 19:20:59.210274 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.210258 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j684c\"" Jul 06 19:20:59.223957 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.223920 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-h5pwl" podStartSLOduration=45.329644534 podStartE2EDuration="50.223909238s" totalImagesPullingTime="4.894264704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:59.223251269 +0000 UTC m=+73.903285065" watchObservedRunningTime="2026-07-06 19:20:59.223909238 +0000 UTC m=+73.903943074" Jul 06 19:20:59.243971 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.243909 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n97st" podStartSLOduration=33.512051741 podStartE2EDuration="38.24389632s" totalImagesPullingTime="4.731844579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:59.24253689 +0000 UTC m=+73.922570688" watchObservedRunningTime="2026-07-06 19:20:59.24389632 +0000 UTC m=+73.923930108" Jul 06 19:20:59.261570 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.261537 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4z89j" podStartSLOduration=45.121637364 podStartE2EDuration="50.261525394s" totalImagesPullingTime="5.13988803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:59.260291782 +0000 UTC m=+73.940325569" watchObservedRunningTime="2026-07-06 19:20:59.261525394 +0000 UTC m=+73.941559190" Jul 06 19:20:59.309646 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.309619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gjh44\" (UID: \"908a4923-8ce3-4024-9587-31b9851a4c6c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:20:59.410385 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.410358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gjh44\" (UID: \"908a4923-8ce3-4024-9587-31b9851a4c6c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:20:59.410501 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:59.410487 2576 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 19:20:59.410555 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:20:59.410545 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates podName:908a4923-8ce3-4024-9587-31b9851a4c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:20:59.910529583 +0000 UTC m=+74.590563358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-gjh44" (UID: "908a4923-8ce3-4024-9587-31b9851a4c6c") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 19:20:59.913678 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.913645 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gjh44\" (UID: \"908a4923-8ce3-4024-9587-31b9851a4c6c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:20:59.916583 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:20:59.916556 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/908a4923-8ce3-4024-9587-31b9851a4c6c-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gjh44\" (UID: \"908a4923-8ce3-4024-9587-31b9851a4c6c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:21:00.116951 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:00.116910 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:21:00.324720 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:00.324689 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44"] Jul 06 19:21:00.329459 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:21:00.329429 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908a4923_8ce3_4024_9587_31b9851a4c6c.slice/crio-d79322513aad7b0fff548579c5c7ca74e84c9c95e894f81b67a6a34579c60d5e WatchSource:0}: Error finding container d79322513aad7b0fff548579c5c7ca74e84c9c95e894f81b67a6a34579c60d5e: Status 404 returned error can't find the container with id d79322513aad7b0fff548579c5c7ca74e84c9c95e894f81b67a6a34579c60d5e Jul 06 19:21:01.197491 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:01.197339 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" event={"ID":"908a4923-8ce3-4024-9587-31b9851a4c6c","Type":"ContainerStarted","Data":"d79322513aad7b0fff548579c5c7ca74e84c9c95e894f81b67a6a34579c60d5e"} Jul 06 19:21:03.207976 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:03.207907 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" event={"ID":"a987b070-f4fe-4af1-8564-6f2ae0be7eaa","Type":"ContainerStarted","Data":"f79ab7afb32ff5156903cd144caa0353d295805ba96ae38ca48f7f3c8c3ed864"} Jul 06 19:21:03.225663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:03.225614 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569f7f8f88-j9zp6" podStartSLOduration=1.395734287 podStartE2EDuration="5.2255984s" totalImagesPullingTime="3.829864113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:21:03.225340221 +0000 UTC m=+77.905374019" watchObservedRunningTime="2026-07-06 19:21:03.2255984 +0000 UTC m=+77.905632196" Jul 06 19:21:05.161548 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.161454 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:21:05.163884 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.163861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nz4z\" (UID: \"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26\") " pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:21:05.216618 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.216580 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" event={"ID":"908a4923-8ce3-4024-9587-31b9851a4c6c","Type":"ContainerStarted","Data":"2a2f13e9691215c3c26ece93dc056909d7c885dbe63356e53b7c37cadf237c8c"} Jul 06 19:21:05.216775 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.216750 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:21:05.218187 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.218160 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" event={"ID":"86b2b1d9-e13b-4515-84a6-243ec5129f4c","Type":"ContainerStarted","Data":"e9c53705e994e6f4cf163b8e8aa5c2e83272366942065c2fca69de8dfceafb15"} Jul 06 19:21:05.218375 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.218359 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:21:05.219872 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.219854 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" Jul 06 19:21:05.221874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.221856 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" Jul 06 19:21:05.233849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.233809 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gjh44" podStartSLOduration=2.112673435 podStartE2EDuration="6.233796693s" totalImagesPullingTime="4.121123258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:21:05.233380678 +0000 UTC m=+79.913414475" watchObservedRunningTime="2026-07-06 19:21:05.233796693 +0000 UTC m=+79.913830489" Jul 06 19:21:05.251343 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.251303 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f867765d5-gl6cg" podStartSLOduration=1.519972966 podStartE2EDuration="7.251286833s" totalImagesPullingTime="5.731313867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:21:05.250624719 +0000 UTC m=+79.930658541" watchObservedRunningTime="2026-07-06 19:21:05.251286833 +0000 UTC m=+79.931320629" Jul 06 19:21:05.458903 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.458852 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2lnlz\"" Jul 06 19:21:05.467046 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.467021 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7nz4z" Jul 06 19:21:05.629413 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:05.629372 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7nz4z"] Jul 06 19:21:05.633097 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:21:05.633070 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aaf41ad_67fc_4fbc_ae11_af6a5ae0ab26.slice/crio-49db54c39f1c88e6c0bd723cedbd1741f34649da2637ac30e9db92e4e94f0bfa WatchSource:0}: Error finding container 49db54c39f1c88e6c0bd723cedbd1741f34649da2637ac30e9db92e4e94f0bfa: Status 404 returned error can't find the container with id 49db54c39f1c88e6c0bd723cedbd1741f34649da2637ac30e9db92e4e94f0bfa Jul 06 19:21:06.228454 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:06.228412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nz4z" event={"ID":"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26","Type":"ContainerStarted","Data":"49db54c39f1c88e6c0bd723cedbd1741f34649da2637ac30e9db92e4e94f0bfa"} Jul 06 19:21:10.200618 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:10.200590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n97st" Jul 06 19:21:10.247727 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:10.247694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nz4z" event={"ID":"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26","Type":"ContainerStarted","Data":"797aca8ef9820b611b6cdb4bd87313e7a3914570a6a768e6102855257241c28c"} Jul 06 19:21:10.247833 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:10.247767 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nz4z" event={"ID":"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26","Type":"ContainerStarted","Data":"0437aded8a61f7e8959f9aa7b808ce673255553fe91d26c53fc0984d52892a59"} Jul 06 19:21:13.262175 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:13.262140 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nz4z" event={"ID":"0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26","Type":"ContainerStarted","Data":"cb70edf2e494553c967a8c50e1d89c0b6233d6f6c6c3cfd8062dc22359a64b2c"} Jul 06 19:21:13.281207 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:13.281170 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7nz4z" podStartSLOduration=33.490734385 podStartE2EDuration="40.281155069s" totalImagesPullingTime="6.790420684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:20:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:21:13.280160975 +0000 UTC m=+87.960194766" watchObservedRunningTime="2026-07-06 19:21:13.281155069 +0000 UTC m=+87.961188862" Jul 06 19:21:15.165429 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.165400 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:21:15.676727 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.676695 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2hxgz"] Jul 06 19:21:15.682277 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.682246 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.685401 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.685379 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8s9tm\"" Jul 06 19:21:15.685568 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.685509 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:21:15.685663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.685646 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:21:15.685791 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.685761 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:21:15.686530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.686511 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:21:15.743694 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-tls\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743848 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743848 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-sys\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743848 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-wtmp\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743846 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-root\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-textfile\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743910 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-metrics-client-ca\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.743961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.743931 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trzx9\" (UniqueName: \"kubernetes.io/projected/051e7ed2-98de-4766-89bd-295206adf997-kube-api-access-trzx9\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845025 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.844984 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-tls\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845227 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845227 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845116 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-sys\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845227 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845185 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-sys\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845236 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-wtmp\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845272 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-root\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-textfile\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845334 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-metrics-client-ca\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845632 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845414 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-wtmp\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845632 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845468 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/051e7ed2-98de-4766-89bd-295206adf997-root\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845632 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trzx9\" (UniqueName: \"kubernetes.io/projected/051e7ed2-98de-4766-89bd-295206adf997-kube-api-access-trzx9\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845764 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845653 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-textfile\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845814 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845798 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-accelerators-collector-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.845898 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.845880 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/051e7ed2-98de-4766-89bd-295206adf997-metrics-client-ca\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.847654 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.847631 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-tls\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.847791 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.847744 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/051e7ed2-98de-4766-89bd-295206adf997-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.855006 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.854985 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trzx9\" (UniqueName: \"kubernetes.io/projected/051e7ed2-98de-4766-89bd-295206adf997-kube-api-access-trzx9\") pod \"node-exporter-2hxgz\" (UID: \"051e7ed2-98de-4766-89bd-295206adf997\") " pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:15.990898 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:15.990874 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2hxgz" Jul 06 19:21:16.003073 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:21:16.003037 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051e7ed2_98de_4766_89bd_295206adf997.slice/crio-373476f3abbce186ae3b79c61675a6cb93eb2a36cac634b20a2800a37d8940b4 WatchSource:0}: Error finding container 373476f3abbce186ae3b79c61675a6cb93eb2a36cac634b20a2800a37d8940b4: Status 404 returned error can't find the container with id 373476f3abbce186ae3b79c61675a6cb93eb2a36cac634b20a2800a37d8940b4 Jul 06 19:21:16.272881 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:16.272801 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hxgz" event={"ID":"051e7ed2-98de-4766-89bd-295206adf997","Type":"ContainerStarted","Data":"373476f3abbce186ae3b79c61675a6cb93eb2a36cac634b20a2800a37d8940b4"} Jul 06 19:21:17.278023 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:17.277991 2576 generic.go:356] "Generic (PLEG): container finished" podID="051e7ed2-98de-4766-89bd-295206adf997" containerID="0491d5738e4e2650aae5c1b234d51cca59081b85abab11b7790d7d2fe18f0652" exitCode=0 Jul 06 19:21:17.278374 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:17.278081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hxgz" event={"ID":"051e7ed2-98de-4766-89bd-295206adf997","Type":"ContainerDied","Data":"0491d5738e4e2650aae5c1b234d51cca59081b85abab11b7790d7d2fe18f0652"} Jul 06 19:21:18.287424 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:18.287388 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hxgz" event={"ID":"051e7ed2-98de-4766-89bd-295206adf997","Type":"ContainerStarted","Data":"9182a4559e8ab484445ca9231b81b54ab1d2cae1a30f653000d575e1a5933b42"} Jul 06 19:21:18.287860 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:18.287473 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2hxgz" event={"ID":"051e7ed2-98de-4766-89bd-295206adf997","Type":"ContainerStarted","Data":"cc239e4db8b326700200a88add8c249a741605257f4a42dd8876f4f92a707df5"} Jul 06 19:21:18.315405 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:18.315361 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2hxgz" podStartSLOduration=2.179491995 podStartE2EDuration="3.31534709s" totalImagesPullingTime="1.070364736s" totalInitContainerRuntime="65.490359ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:21:18.315029467 +0000 UTC m=+92.995063263" watchObservedRunningTime="2026-07-06 19:21:18.31534709 +0000 UTC m=+92.995380886" Jul 06 19:21:20.467242 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:20.467207 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:21:23.143595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:23.143567 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s2wf7" Jul 06 19:21:37.366293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:37.366257 2576 generic.go:356] "Generic (PLEG): container finished" podID="134871dc-e7c2-4a57-a4c4-2fa19ddf5aed" containerID="79c9fd16308e2793ab62edeb6e70af49d8e2a5a151b3ecb19d9115745ead189f" exitCode=0 Jul 06 19:21:37.366665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:37.366330 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" event={"ID":"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed","Type":"ContainerDied","Data":"79c9fd16308e2793ab62edeb6e70af49d8e2a5a151b3ecb19d9115745ead189f"} Jul 06 19:21:37.366665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:37.366652 2576 scope.go:117] "RemoveContainer" containerID="79c9fd16308e2793ab62edeb6e70af49d8e2a5a151b3ecb19d9115745ead189f" Jul 06 19:21:38.371721 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:38.371688 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-gqdvx" event={"ID":"134871dc-e7c2-4a57-a4c4-2fa19ddf5aed","Type":"ContainerStarted","Data":"096c8a2b4ec48794dcbbc6a9a92a79ae8f6f65414c27079ecc09fe22566d18eb"} Jul 06 19:21:45.487064 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.487014 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" podUID="b8d93d2c-cf57-4578-9195-916e6987a547" containerName="registry" containerID="cri-o://7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6" gracePeriod=30 Jul 06 19:21:45.731562 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.731540 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:21:45.882880 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.882810 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.882880 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.882845 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.882880 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.882870 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883121 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.882887 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883121 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.882902 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883121 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883050 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883121 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883099 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gdb7\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883373 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883314 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:21:45.883453 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883426 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:21:45.883514 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883435 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets\") pod \"b8d93d2c-cf57-4578-9195-916e6987a547\" (UID: \"b8d93d2c-cf57-4578-9195-916e6987a547\") " Jul 06 19:21:45.883738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883695 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-registry-certificates\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.883738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.883721 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8d93d2c-cf57-4578-9195-916e6987a547-trusted-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.885477 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.885448 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:21:45.885605 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.885546 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:21:45.885605 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.885563 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:21:45.885728 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.885639 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7" (OuterVolumeSpecName: "kube-api-access-8gdb7") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "kube-api-access-8gdb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:21:45.885784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.885727 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:21:45.890789 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.890760 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b8d93d2c-cf57-4578-9195-916e6987a547" (UID: "b8d93d2c-cf57-4578-9195-916e6987a547"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:21:45.984418 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984392 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-registry-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.984418 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984419 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8d93d2c-cf57-4578-9195-916e6987a547-ca-trust-extracted\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.984592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984433 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-bound-sa-token\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.984592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984447 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-image-registry-private-configuration\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.984592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984461 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gdb7\" (UniqueName: \"kubernetes.io/projected/b8d93d2c-cf57-4578-9195-916e6987a547-kube-api-access-8gdb7\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:45.984592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:45.984472 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8d93d2c-cf57-4578-9195-916e6987a547-installation-pull-secrets\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:21:46.405874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.405833 2576 generic.go:356] "Generic (PLEG): container finished" podID="b8d93d2c-cf57-4578-9195-916e6987a547" containerID="7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6" exitCode=0 Jul 06 19:21:46.406068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.405882 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" Jul 06 19:21:46.406068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.405921 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" event={"ID":"b8d93d2c-cf57-4578-9195-916e6987a547","Type":"ContainerDied","Data":"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6"} Jul 06 19:21:46.406068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.405972 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d8d6779cf-rgklz" event={"ID":"b8d93d2c-cf57-4578-9195-916e6987a547","Type":"ContainerDied","Data":"1d16f7bff4b1783e9f5608ee2b5701b4c861d4874de2512e42645b78fe3036dd"} Jul 06 19:21:46.406068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.405995 2576 scope.go:117] "RemoveContainer" containerID="7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6" Jul 06 19:21:46.415160 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.415111 2576 scope.go:117] "RemoveContainer" containerID="7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6" Jul 06 19:21:46.415403 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:21:46.415384 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6\": container with ID starting with 7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6 not found: ID does not exist" containerID="7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6" Jul 06 19:21:46.415448 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.415412 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6"} err="failed to get container status \"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6\": rpc error: code = NotFound desc = could not find container \"7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6\": container with ID starting with 7ada2cf03a4c918573e9906d05e544252e5be1ae574b11b16c41acef4220cce6 not found: ID does not exist" Jul 06 19:21:46.427678 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.427656 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:21:46.433976 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:46.433959 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d8d6779cf-rgklz"] Jul 06 19:21:47.848665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:47.848629 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d93d2c-cf57-4578-9195-916e6987a547" path="/var/lib/kubelet/pods/b8d93d2c-cf57-4578-9195-916e6987a547/volumes" Jul 06 19:21:53.432938 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:53.432905 2576 generic.go:356] "Generic (PLEG): container finished" podID="9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65" containerID="c0db11153d8245caf9362f694996770164f5c001e3104b652bea9cf4d1ec2020" exitCode=0 Jul 06 19:21:53.433352 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:53.432959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" event={"ID":"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65","Type":"ContainerDied","Data":"c0db11153d8245caf9362f694996770164f5c001e3104b652bea9cf4d1ec2020"} Jul 06 19:21:53.433352 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:53.433299 2576 scope.go:117] "RemoveContainer" containerID="c0db11153d8245caf9362f694996770164f5c001e3104b652bea9cf4d1ec2020" Jul 06 19:21:54.442121 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:21:54.442084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-gdrrs" event={"ID":"9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65","Type":"ContainerStarted","Data":"9a3d01e2d2f02ffe02ec6bf3ad2bda9eed470d083f2de24ae7b6137b52cf1234"} Jul 06 19:22:07.503397 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:22:07.503363 2576 generic.go:356] "Generic (PLEG): container finished" podID="f44fb294-1867-4650-a8c8-1a7d7cf0378b" containerID="5604692838fd242ddd3922a1c8eca2d12e2656d2c75facbd876166992eaa180c" exitCode=0 Jul 06 19:22:07.503767 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:22:07.503436 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" event={"ID":"f44fb294-1867-4650-a8c8-1a7d7cf0378b","Type":"ContainerDied","Data":"5604692838fd242ddd3922a1c8eca2d12e2656d2c75facbd876166992eaa180c"} Jul 06 19:22:07.503767 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:22:07.503745 2576 scope.go:117] "RemoveContainer" containerID="5604692838fd242ddd3922a1c8eca2d12e2656d2c75facbd876166992eaa180c" Jul 06 19:22:08.508732 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:22:08.508700 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wm6gc" event={"ID":"f44fb294-1867-4650-a8c8-1a7d7cf0378b","Type":"ContainerStarted","Data":"eaaf0b57534dd09e3ef4febce289d8cef3ead0bfe1e8d123b9d26ced7614772f"} Jul 06 19:24:30.466445 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.466410 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk"] Jul 06 19:24:30.466898 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.466679 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d93d2c-cf57-4578-9195-916e6987a547" containerName="registry" Jul 06 19:24:30.466898 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.466691 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d93d2c-cf57-4578-9195-916e6987a547" containerName="registry" Jul 06 19:24:30.466898 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.466744 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d93d2c-cf57-4578-9195-916e6987a547" containerName="registry" Jul 06 19:24:30.468522 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.468507 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.472663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.472644 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:24:30.472945 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.472929 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:24:30.474242 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.474224 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7gbdk\"" Jul 06 19:24:30.483593 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.483576 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk"] Jul 06 19:24:30.594794 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.594763 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.594976 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.594842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48ldf\" (UniqueName: \"kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.594976 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.594893 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.695729 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.695700 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.695849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.695759 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48ldf\" (UniqueName: \"kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.695849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.695783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.696064 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.696046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.696103 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.696074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.705706 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.705688 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48ldf\" (UniqueName: \"kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.777399 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.777347 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:30.906747 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:30.906672 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk"] Jul 06 19:24:30.909295 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:24:30.909261 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a133138_9d05_4341_9c3a_86a39d790610.slice/crio-38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6 WatchSource:0}: Error finding container 38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6: Status 404 returned error can't find the container with id 38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6 Jul 06 19:24:31.014193 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:31.014156 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerStarted","Data":"38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6"} Jul 06 19:24:39.052543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:39.052515 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a133138-9d05-4341-9c3a-86a39d790610" containerID="bd37e71b4cb40b80abafb946fe5199a54920801a12249db06c7935114805dc64" exitCode=0 Jul 06 19:24:39.052926 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:39.052605 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerDied","Data":"bd37e71b4cb40b80abafb946fe5199a54920801a12249db06c7935114805dc64"} Jul 06 19:24:41.061190 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:41.061089 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a133138-9d05-4341-9c3a-86a39d790610" containerID="10bc26a33efb4a257265f0ca516b7899224fa832ba524610fee8a03aa2a5c145" exitCode=0 Jul 06 19:24:41.061190 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:41.061171 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerDied","Data":"10bc26a33efb4a257265f0ca516b7899224fa832ba524610fee8a03aa2a5c145"} Jul 06 19:24:45.676125 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:45.676092 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:24:45.679619 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:45.679589 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:24:45.690060 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:45.690040 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:24:49.093089 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:49.093055 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerStarted","Data":"a013e3036df7b5423499725b1a2009bedd1ea219cb3451311c48494c87835e07"} Jul 06 19:24:49.114956 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:49.114911 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" podStartSLOduration=1.765124863 podStartE2EDuration="19.114895158s" totalImagesPullingTime="17.349770295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:49.113414898 +0000 UTC m=+303.793448696" watchObservedRunningTime="2026-07-06 19:24:49.114895158 +0000 UTC m=+303.794928955" Jul 06 19:24:50.098928 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:50.098895 2576 generic.go:356] "Generic (PLEG): container finished" podID="4a133138-9d05-4341-9c3a-86a39d790610" containerID="a013e3036df7b5423499725b1a2009bedd1ea219cb3451311c48494c87835e07" exitCode=0 Jul 06 19:24:50.099336 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:50.098967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerDied","Data":"a013e3036df7b5423499725b1a2009bedd1ea219cb3451311c48494c87835e07"} Jul 06 19:24:51.220856 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.220834 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:51.373659 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.373558 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle\") pod \"4a133138-9d05-4341-9c3a-86a39d790610\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " Jul 06 19:24:51.373659 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.373607 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48ldf\" (UniqueName: \"kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf\") pod \"4a133138-9d05-4341-9c3a-86a39d790610\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " Jul 06 19:24:51.373848 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.373665 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util\") pod \"4a133138-9d05-4341-9c3a-86a39d790610\" (UID: \"4a133138-9d05-4341-9c3a-86a39d790610\") " Jul 06 19:24:51.374252 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.374219 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle" (OuterVolumeSpecName: "bundle") pod "4a133138-9d05-4341-9c3a-86a39d790610" (UID: "4a133138-9d05-4341-9c3a-86a39d790610"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:24:51.375811 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.375786 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf" (OuterVolumeSpecName: "kube-api-access-48ldf") pod "4a133138-9d05-4341-9c3a-86a39d790610" (UID: "4a133138-9d05-4341-9c3a-86a39d790610"). InnerVolumeSpecName "kube-api-access-48ldf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:24:51.377877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.377858 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util" (OuterVolumeSpecName: "util") pod "4a133138-9d05-4341-9c3a-86a39d790610" (UID: "4a133138-9d05-4341-9c3a-86a39d790610"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:24:51.474101 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.474072 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:24:51.474101 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.474094 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48ldf\" (UniqueName: \"kubernetes.io/projected/4a133138-9d05-4341-9c3a-86a39d790610-kube-api-access-48ldf\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:24:51.474101 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:51.474103 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a133138-9d05-4341-9c3a-86a39d790610-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:24:52.108050 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:52.107983 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" Jul 06 19:24:52.108050 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:52.107997 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540b86tk" event={"ID":"4a133138-9d05-4341-9c3a-86a39d790610","Type":"ContainerDied","Data":"38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6"} Jul 06 19:24:52.108050 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:52.108040 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38dae0eaea4f557bbcb684e84f7435350b72761bbdb7d39fb82068eaf8e3e2e6" Jul 06 19:24:59.028792 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.028753 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9"] Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029170 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="extract" Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029188 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="extract" Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029203 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="util" Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029211 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="util" Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029234 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="pull" Jul 06 19:24:59.029283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029242 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="pull" Jul 06 19:24:59.029564 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.029326 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a133138-9d05-4341-9c3a-86a39d790610" containerName="extract" Jul 06 19:24:59.032755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.032736 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.036304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.036282 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:24:59.036540 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.036524 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 19:24:59.036760 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.036747 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4fd6b\"" Jul 06 19:24:59.046552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.046529 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9"] Jul 06 19:24:59.126580 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.126548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00ee6f5c-ea39-410f-8136-de767dcc6c46-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.126753 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.126580 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qz2x\" (UniqueName: \"kubernetes.io/projected/00ee6f5c-ea39-410f-8136-de767dcc6c46-kube-api-access-5qz2x\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.227641 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.227611 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00ee6f5c-ea39-410f-8136-de767dcc6c46-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.227641 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.227643 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qz2x\" (UniqueName: \"kubernetes.io/projected/00ee6f5c-ea39-410f-8136-de767dcc6c46-kube-api-access-5qz2x\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.227967 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.227947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00ee6f5c-ea39-410f-8136-de767dcc6c46-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.240838 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.240811 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qz2x\" (UniqueName: \"kubernetes.io/projected/00ee6f5c-ea39-410f-8136-de767dcc6c46-kube-api-access-5qz2x\") pod \"cert-manager-operator-controller-manager-55c444bf86-75nt9\" (UID: \"00ee6f5c-ea39-410f-8136-de767dcc6c46\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.342251 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.342188 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" Jul 06 19:24:59.476804 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.476772 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9"] Jul 06 19:24:59.480990 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:24:59.480963 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ee6f5c_ea39_410f_8136_de767dcc6c46.slice/crio-b492fc78cc4f22254e5d8067680e1805b08b4de9d33634d7a095a3305a4ecd2e WatchSource:0}: Error finding container b492fc78cc4f22254e5d8067680e1805b08b4de9d33634d7a095a3305a4ecd2e: Status 404 returned error can't find the container with id b492fc78cc4f22254e5d8067680e1805b08b4de9d33634d7a095a3305a4ecd2e Jul 06 19:24:59.484463 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:24:59.484446 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:25:00.137962 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:00.137928 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" event={"ID":"00ee6f5c-ea39-410f-8136-de767dcc6c46","Type":"ContainerStarted","Data":"b492fc78cc4f22254e5d8067680e1805b08b4de9d33634d7a095a3305a4ecd2e"} Jul 06 19:25:02.147884 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.147851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" event={"ID":"00ee6f5c-ea39-410f-8136-de767dcc6c46","Type":"ContainerStarted","Data":"1fd93b0655056070f9896986b901babe72e4adf28aaa586c1e6d41c25b234d3b"} Jul 06 19:25:02.176298 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.176251 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-75nt9" podStartSLOduration=1.455100192 podStartE2EDuration="3.17623678s" totalImagesPullingTime="1.721136588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:02.1736921 +0000 UTC m=+316.853725898" watchObservedRunningTime="2026-07-06 19:25:02.17623678 +0000 UTC m=+316.856270578" Jul 06 19:25:02.972593 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.972565 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5"] Jul 06 19:25:02.974915 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.974897 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:02.978232 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.978212 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7gbdk\"" Jul 06 19:25:02.982000 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.981978 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:25:02.982949 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:02.982931 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:25:03.001641 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.001618 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5"] Jul 06 19:25:03.055558 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.055531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.055663 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.055593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.055741 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.055686 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5d9k\" (UniqueName: \"kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.156621 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.156592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.157048 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.156626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5d9k\" (UniqueName: \"kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.157048 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.156812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.157048 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.156985 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.157200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.157051 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.170598 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.170568 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5d9k\" (UniqueName: \"kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.283845 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.283773 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:03.414406 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:03.414382 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5"] Jul 06 19:25:03.416596 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:03.416570 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6448a9b9_b932_48be_b126_124d8325e19b.slice/crio-8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c WatchSource:0}: Error finding container 8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c: Status 404 returned error can't find the container with id 8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c Jul 06 19:25:04.157635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:04.157601 2576 generic.go:356] "Generic (PLEG): container finished" podID="6448a9b9-b932-48be-b126-124d8325e19b" containerID="63b37c85ee55da883c93fc3a10c0feb9e2cd96472d08cbd2b422c2231e5c3277" exitCode=0 Jul 06 19:25:04.158041 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:04.157694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" event={"ID":"6448a9b9-b932-48be-b126-124d8325e19b","Type":"ContainerDied","Data":"63b37c85ee55da883c93fc3a10c0feb9e2cd96472d08cbd2b422c2231e5c3277"} Jul 06 19:25:04.158041 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:04.157762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" event={"ID":"6448a9b9-b932-48be-b126-124d8325e19b","Type":"ContainerStarted","Data":"8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c"} Jul 06 19:25:06.168364 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:06.168277 2576 generic.go:356] "Generic (PLEG): container finished" podID="6448a9b9-b932-48be-b126-124d8325e19b" containerID="81fda08e247bcffb6e8c9ef9be1cc074c038590a7aec1e244b056514e5bd0654" exitCode=0 Jul 06 19:25:06.168698 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:06.168365 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" event={"ID":"6448a9b9-b932-48be-b126-124d8325e19b","Type":"ContainerDied","Data":"81fda08e247bcffb6e8c9ef9be1cc074c038590a7aec1e244b056514e5bd0654"} Jul 06 19:25:07.174305 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:07.174268 2576 generic.go:356] "Generic (PLEG): container finished" podID="6448a9b9-b932-48be-b126-124d8325e19b" containerID="e85639bd96db229daac934ed133b337c9f25cad364770722a1c79562d0576f62" exitCode=0 Jul 06 19:25:07.174722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:07.174338 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" event={"ID":"6448a9b9-b932-48be-b126-124d8325e19b","Type":"ContainerDied","Data":"e85639bd96db229daac934ed133b337c9f25cad364770722a1c79562d0576f62"} Jul 06 19:25:08.301229 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.301207 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:08.397590 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.397562 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util\") pod \"6448a9b9-b932-48be-b126-124d8325e19b\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " Jul 06 19:25:08.397738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.397604 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle\") pod \"6448a9b9-b932-48be-b126-124d8325e19b\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " Jul 06 19:25:08.397738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.397644 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5d9k\" (UniqueName: \"kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k\") pod \"6448a9b9-b932-48be-b126-124d8325e19b\" (UID: \"6448a9b9-b932-48be-b126-124d8325e19b\") " Jul 06 19:25:08.398090 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.398043 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle" (OuterVolumeSpecName: "bundle") pod "6448a9b9-b932-48be-b126-124d8325e19b" (UID: "6448a9b9-b932-48be-b126-124d8325e19b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:08.399820 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.399795 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k" (OuterVolumeSpecName: "kube-api-access-w5d9k") pod "6448a9b9-b932-48be-b126-124d8325e19b" (UID: "6448a9b9-b932-48be-b126-124d8325e19b"). InnerVolumeSpecName "kube-api-access-w5d9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:25:08.405660 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.405636 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util" (OuterVolumeSpecName: "util") pod "6448a9b9-b932-48be-b126-124d8325e19b" (UID: "6448a9b9-b932-48be-b126-124d8325e19b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:08.498419 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.498393 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5d9k\" (UniqueName: \"kubernetes.io/projected/6448a9b9-b932-48be-b126-124d8325e19b-kube-api-access-w5d9k\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:08.498419 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.498415 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:08.498573 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:08.498425 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6448a9b9-b932-48be-b126-124d8325e19b-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:09.183569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.183534 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" event={"ID":"6448a9b9-b932-48be-b126-124d8325e19b","Type":"ContainerDied","Data":"8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c"} Jul 06 19:25:09.183569 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.183566 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z7g5" Jul 06 19:25:09.183771 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.183579 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f14eead8f9b3c504b824b8233d43ab6b271016f7e859c13f5d757378abd4a0c" Jul 06 19:25:09.685236 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685207 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cxrdq"] Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685471 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="util" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685482 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="util" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685489 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="pull" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685495 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="pull" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685510 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="extract" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685516 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="extract" Jul 06 19:25:09.685635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.685578 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6448a9b9-b932-48be-b126-124d8325e19b" containerName="extract" Jul 06 19:25:09.688182 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.688164 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.691107 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.691087 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:25:09.692038 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.692024 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ccxhw\"" Jul 06 19:25:09.692095 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.692038 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:25:09.699198 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.699179 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cxrdq"] Jul 06 19:25:09.704948 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.704921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9nxd\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-kube-api-access-x9nxd\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.705043 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.705015 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.805712 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.805668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.805904 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.805751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9nxd\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-kube-api-access-x9nxd\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.815206 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.815170 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.815318 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.815296 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9nxd\" (UniqueName: \"kubernetes.io/projected/63946b31-fd0c-4a82-883d-82d8886a36d0-kube-api-access-x9nxd\") pod \"cert-manager-webhook-6486ffd997-cxrdq\" (UID: \"63946b31-fd0c-4a82-883d-82d8886a36d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:09.997210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:09.997116 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:10.124815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:10.124784 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cxrdq"] Jul 06 19:25:10.128398 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:10.128368 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63946b31_fd0c_4a82_883d_82d8886a36d0.slice/crio-aab218f778e6062fe1398f1e82b0d2ce7b320125d60a8e01de245451a0d6ea2f WatchSource:0}: Error finding container aab218f778e6062fe1398f1e82b0d2ce7b320125d60a8e01de245451a0d6ea2f: Status 404 returned error can't find the container with id aab218f778e6062fe1398f1e82b0d2ce7b320125d60a8e01de245451a0d6ea2f Jul 06 19:25:10.189168 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:10.189111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" event={"ID":"63946b31-fd0c-4a82-883d-82d8886a36d0","Type":"ContainerStarted","Data":"aab218f778e6062fe1398f1e82b0d2ce7b320125d60a8e01de245451a0d6ea2f"} Jul 06 19:25:19.234978 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:19.234940 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" event={"ID":"63946b31-fd0c-4a82-883d-82d8886a36d0","Type":"ContainerStarted","Data":"754bc262ca0f061a7c5f487c15befcaa17aedf093c399d63aab728dce39fc639"} Jul 06 19:25:19.235363 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:19.235064 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:19.257283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:19.257232 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" podStartSLOduration=1.639025367 podStartE2EDuration="10.257217155s" totalImagesPullingTime="8.618191788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:19.255545023 +0000 UTC m=+333.935578811" watchObservedRunningTime="2026-07-06 19:25:19.257217155 +0000 UTC m=+333.937250952" Jul 06 19:25:20.134302 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.134262 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd"] Jul 06 19:25:20.137920 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.137896 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.140904 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.140868 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:25:20.141728 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.141697 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:25:20.141728 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.141697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7gbdk\"" Jul 06 19:25:20.148561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.148531 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd"] Jul 06 19:25:20.186808 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.186780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6vww\" (UniqueName: \"kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.186922 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.186822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.186922 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.186860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.287736 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.287711 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.288197 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.287768 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.288197 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.287826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6vww\" (UniqueName: \"kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.288313 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.288216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.288313 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.288222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.297158 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.297117 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6vww\" (UniqueName: \"kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.447258 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.447235 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:20.583469 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:20.583446 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd"] Jul 06 19:25:20.585397 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:20.585368 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86df97a3_4c21_4a12_a23e_6be8e4547034.slice/crio-49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e WatchSource:0}: Error finding container 49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e: Status 404 returned error can't find the container with id 49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e Jul 06 19:25:21.244922 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:21.244888 2576 generic.go:356] "Generic (PLEG): container finished" podID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerID="3888185574aae18aa910176e07d71107253276d4870638cbc3b37e3bfafec877" exitCode=0 Jul 06 19:25:21.245104 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:21.244976 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerDied","Data":"3888185574aae18aa910176e07d71107253276d4870638cbc3b37e3bfafec877"} Jul 06 19:25:21.245104 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:21.245025 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerStarted","Data":"49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e"} Jul 06 19:25:22.251694 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:22.251660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerStarted","Data":"26e52a7fa2f29b4b58f505844f83405624d89577b89890a15f9e731d814bbf73"} Jul 06 19:25:23.257597 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:23.257563 2576 generic.go:356] "Generic (PLEG): container finished" podID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerID="26e52a7fa2f29b4b58f505844f83405624d89577b89890a15f9e731d814bbf73" exitCode=0 Jul 06 19:25:23.257951 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:23.257646 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerDied","Data":"26e52a7fa2f29b4b58f505844f83405624d89577b89890a15f9e731d814bbf73"} Jul 06 19:25:24.263841 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:24.263801 2576 generic.go:356] "Generic (PLEG): container finished" podID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerID="ff7fb74479f166f7836ad7f7bc81f6ea6a7e995cc7d015127d887f4a9ced24e0" exitCode=0 Jul 06 19:25:24.264303 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:24.263866 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerDied","Data":"ff7fb74479f166f7836ad7f7bc81f6ea6a7e995cc7d015127d887f4a9ced24e0"} Jul 06 19:25:25.241667 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.241637 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cxrdq" Jul 06 19:25:25.389609 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.389587 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:25.430507 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.430484 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util\") pod \"86df97a3-4c21-4a12-a23e-6be8e4547034\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " Jul 06 19:25:25.430625 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.430539 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle\") pod \"86df97a3-4c21-4a12-a23e-6be8e4547034\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " Jul 06 19:25:25.430625 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.430598 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6vww\" (UniqueName: \"kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww\") pod \"86df97a3-4c21-4a12-a23e-6be8e4547034\" (UID: \"86df97a3-4c21-4a12-a23e-6be8e4547034\") " Jul 06 19:25:25.431297 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.431253 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle" (OuterVolumeSpecName: "bundle") pod "86df97a3-4c21-4a12-a23e-6be8e4547034" (UID: "86df97a3-4c21-4a12-a23e-6be8e4547034"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:25.432587 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.432557 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww" (OuterVolumeSpecName: "kube-api-access-k6vww") pod "86df97a3-4c21-4a12-a23e-6be8e4547034" (UID: "86df97a3-4c21-4a12-a23e-6be8e4547034"). InnerVolumeSpecName "kube-api-access-k6vww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:25:25.435724 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.435705 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util" (OuterVolumeSpecName: "util") pod "86df97a3-4c21-4a12-a23e-6be8e4547034" (UID: "86df97a3-4c21-4a12-a23e-6be8e4547034"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:25:25.532017 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.531949 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:25.532017 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.531971 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86df97a3-4c21-4a12-a23e-6be8e4547034-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:25.532017 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:25.531982 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6vww\" (UniqueName: \"kubernetes.io/projected/86df97a3-4c21-4a12-a23e-6be8e4547034-kube-api-access-k6vww\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:25:26.273886 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:26.273855 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" event={"ID":"86df97a3-4c21-4a12-a23e-6be8e4547034","Type":"ContainerDied","Data":"49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e"} Jul 06 19:25:26.274052 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:26.273891 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhsjhd" Jul 06 19:25:26.274052 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:26.273900 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49a7037e12ab6b571c7b388225c31cfcb7b4327d47b26379061f89cc7e32867e" Jul 06 19:25:33.015247 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015215 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7"] Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015513 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="util" Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015524 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="util" Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015540 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="extract" Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015548 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="extract" Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015559 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="pull" Jul 06 19:25:33.015611 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015564 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="pull" Jul 06 19:25:33.015791 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.015623 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="86df97a3-4c21-4a12-a23e-6be8e4547034" containerName="extract" Jul 06 19:25:33.018176 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.018158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.024503 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.024359 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:25:33.024585 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.024559 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tfnth\"" Jul 06 19:25:33.024635 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.024621 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:25:33.024691 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.024672 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:25:33.024815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.024799 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:25:33.058317 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.058285 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7"] Jul 06 19:25:33.092398 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.092377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-tmp\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.092513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.092415 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtjw6\" (UniqueName: \"kubernetes.io/projected/10149d48-7aa3-438d-ad38-a68e33be111d-kube-api-access-jtjw6\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.092513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.092447 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.092579 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.092511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-manifests\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.092579 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.092545 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.192996 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.192970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtjw6\" (UniqueName: \"kubernetes.io/projected/10149d48-7aa3-438d-ad38-a68e33be111d-kube-api-access-jtjw6\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.193120 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.193007 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.193120 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.193041 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-manifests\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.193245 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.193165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.193245 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.193230 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-tmp\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.193489 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.193468 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-manifests\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.195295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.195267 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10149d48-7aa3-438d-ad38-a68e33be111d-tmp\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.195549 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.195529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.195657 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.195634 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10149d48-7aa3-438d-ad38-a68e33be111d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.210651 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.210629 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtjw6\" (UniqueName: \"kubernetes.io/projected/10149d48-7aa3-438d-ad38-a68e33be111d-kube-api-access-jtjw6\") pod \"opendatahub-operator-controller-manager-69fd78d8d5-m5bn7\" (UID: \"10149d48-7aa3-438d-ad38-a68e33be111d\") " pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.328364 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.328293 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:33.480124 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:33.480096 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7"] Jul 06 19:25:33.484090 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:33.484065 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10149d48_7aa3_438d_ad38_a68e33be111d.slice/crio-eb21150475bde5322fd8f242909bf89db76f7929c494cb277f7ec15cfe056041 WatchSource:0}: Error finding container eb21150475bde5322fd8f242909bf89db76f7929c494cb277f7ec15cfe056041: Status 404 returned error can't find the container with id eb21150475bde5322fd8f242909bf89db76f7929c494cb277f7ec15cfe056041 Jul 06 19:25:34.306513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:34.306474 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" event={"ID":"10149d48-7aa3-438d-ad38-a68e33be111d","Type":"ContainerStarted","Data":"eb21150475bde5322fd8f242909bf89db76f7929c494cb277f7ec15cfe056041"} Jul 06 19:25:37.323355 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:37.323318 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" event={"ID":"10149d48-7aa3-438d-ad38-a68e33be111d","Type":"ContainerStarted","Data":"98ab8b40cb72d8cf6e7243de465beaee2e1de46fec5abfcf566d5ca5544433b5"} Jul 06 19:25:38.329355 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:38.329320 2576 generic.go:356] "Generic (PLEG): container finished" podID="10149d48-7aa3-438d-ad38-a68e33be111d" containerID="98ab8b40cb72d8cf6e7243de465beaee2e1de46fec5abfcf566d5ca5544433b5" exitCode=0 Jul 06 19:25:38.329822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:38.329366 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" event={"ID":"10149d48-7aa3-438d-ad38-a68e33be111d","Type":"ContainerDied","Data":"98ab8b40cb72d8cf6e7243de465beaee2e1de46fec5abfcf566d5ca5544433b5"} Jul 06 19:25:39.336143 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:39.336104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" event={"ID":"10149d48-7aa3-438d-ad38-a68e33be111d","Type":"ContainerStarted","Data":"98cb04f58b8ba369e3192616c28576258d68a58eae865b451301146f69ddc235"} Jul 06 19:25:39.336533 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:39.336269 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:39.369248 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:39.369199 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" podStartSLOduration=2.724119904 podStartE2EDuration="7.369182024s" totalImagesPullingTime="3.599928831s" totalInitContainerRuntime="1.045133289s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:39.368504897 +0000 UTC m=+354.048538695" watchObservedRunningTime="2026-07-06 19:25:39.369182024 +0000 UTC m=+354.049215820" Jul 06 19:25:50.342937 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:50.342906 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69fd78d8d5-m5bn7" Jul 06 19:25:53.604836 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.604800 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7"] Jul 06 19:25:53.614504 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.614480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.618745 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.618717 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bcpvz\"" Jul 06 19:25:53.618964 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.618935 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 19:25:53.619090 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.619028 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 19:25:53.621186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.621161 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7"] Jul 06 19:25:53.753104 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.753072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd2z6\" (UniqueName: \"kubernetes.io/projected/fea0f914-328b-4f48-a9db-65a93c985f1b-kube-api-access-nd2z6\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.753293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.753111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea0f914-328b-4f48-a9db-65a93c985f1b-tmp\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.753293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.753201 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea0f914-328b-4f48-a9db-65a93c985f1b-tls-certs\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.764380 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.764343 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5vxj"] Jul 06 19:25:53.767568 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.767546 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:53.771304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.771277 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 19:25:53.780477 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.780457 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wdhln\"" Jul 06 19:25:53.785395 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.785369 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5vxj"] Jul 06 19:25:53.854241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.854208 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd2z6\" (UniqueName: \"kubernetes.io/projected/fea0f914-328b-4f48-a9db-65a93c985f1b-kube-api-access-nd2z6\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.854241 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.854244 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea0f914-328b-4f48-a9db-65a93c985f1b-tmp\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.854452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.854283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w596l\" (UniqueName: \"kubernetes.io/projected/7f818fd7-c436-4d9a-8d78-ffae230e063a-kube-api-access-w596l\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:53.854452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.854336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea0f914-328b-4f48-a9db-65a93c985f1b-tls-certs\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.854452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.854365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:53.856477 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.856413 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea0f914-328b-4f48-a9db-65a93c985f1b-tmp\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.856638 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.856621 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fea0f914-328b-4f48-a9db-65a93c985f1b-tls-certs\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.866488 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.866464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd2z6\" (UniqueName: \"kubernetes.io/projected/fea0f914-328b-4f48-a9db-65a93c985f1b-kube-api-access-nd2z6\") pod \"kube-auth-proxy-7bc75879f5-btxg7\" (UID: \"fea0f914-328b-4f48-a9db-65a93c985f1b\") " pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.926695 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.926654 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" Jul 06 19:25:53.955564 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.955534 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w596l\" (UniqueName: \"kubernetes.io/projected/7f818fd7-c436-4d9a-8d78-ffae230e063a-kube-api-access-w596l\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:53.955706 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.955600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:53.955706 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:25:53.955702 2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 19:25:53.955816 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:25:53.955781 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert podName:7f818fd7-c436-4d9a-8d78-ffae230e063a nodeName:}" failed. No retries permitted until 2026-07-06 19:25:54.455757988 +0000 UTC m=+369.135791784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert") pod "odh-model-controller-7998c67dc4-r5vxj" (UID: "7f818fd7-c436-4d9a-8d78-ffae230e063a") : secret "odh-model-controller-webhook-cert" not found Jul 06 19:25:53.974834 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:53.974803 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w596l\" (UniqueName: \"kubernetes.io/projected/7f818fd7-c436-4d9a-8d78-ffae230e063a-kube-api-access-w596l\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:54.091859 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.091834 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7"] Jul 06 19:25:54.094060 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:54.094030 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea0f914_328b_4f48_a9db_65a93c985f1b.slice/crio-2308d7134fede74501f73afb9fa7e0c333d39d6e3c4a331189c65295421baefd WatchSource:0}: Error finding container 2308d7134fede74501f73afb9fa7e0c333d39d6e3c4a331189c65295421baefd: Status 404 returned error can't find the container with id 2308d7134fede74501f73afb9fa7e0c333d39d6e3c4a331189c65295421baefd Jul 06 19:25:54.404509 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.404471 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" event={"ID":"fea0f914-328b-4f48-a9db-65a93c985f1b","Type":"ContainerStarted","Data":"2308d7134fede74501f73afb9fa7e0c333d39d6e3c4a331189c65295421baefd"} Jul 06 19:25:54.459061 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.459028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:54.459233 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:25:54.459203 2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 19:25:54.459285 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:25:54.459268 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert podName:7f818fd7-c436-4d9a-8d78-ffae230e063a nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.459249835 +0000 UTC m=+370.139283610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert") pod "odh-model-controller-7998c67dc4-r5vxj" (UID: "7f818fd7-c436-4d9a-8d78-ffae230e063a") : secret "odh-model-controller-webhook-cert" not found Jul 06 19:25:54.788001 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.787968 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gqb7x"] Jul 06 19:25:54.791159 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.791123 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:54.794031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.793975 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 19:25:54.794031 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.794015 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xvrrm\"" Jul 06 19:25:54.807433 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.807408 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gqb7x"] Jul 06 19:25:54.963359 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.963320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:54.963529 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:54.963507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxknv\" (UniqueName: \"kubernetes.io/projected/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-kube-api-access-cxknv\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.064588 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.064495 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxknv\" (UniqueName: \"kubernetes.io/projected/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-kube-api-access-cxknv\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.064588 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.064557 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.067049 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.067023 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.077452 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.077428 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxknv\" (UniqueName: \"kubernetes.io/projected/b3052be5-f8ee-4084-a2ca-7b625b79f0f2-kube-api-access-cxknv\") pod \"model-serving-api-54d89bd5fc-gqb7x\" (UID: \"b3052be5-f8ee-4084-a2ca-7b625b79f0f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.105025 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.104996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:25:55.263608 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.263577 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gqb7x"] Jul 06 19:25:55.267478 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:55.267444 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3052be5_f8ee_4084_a2ca_7b625b79f0f2.slice/crio-3dac59ab2b1ed8779a07e48500467378054f2e5b64579a14e0456877376a14e7 WatchSource:0}: Error finding container 3dac59ab2b1ed8779a07e48500467378054f2e5b64579a14e0456877376a14e7: Status 404 returned error can't find the container with id 3dac59ab2b1ed8779a07e48500467378054f2e5b64579a14e0456877376a14e7 Jul 06 19:25:55.319588 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.319511 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:25:55.324003 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.323986 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.331493 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.331472 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:25:55.331719 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.331705 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:25:55.331857 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.331846 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:25:55.332016 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.332004 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8m96c\"" Jul 06 19:25:55.332190 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.332178 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:25:55.346788 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.346764 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:25:55.410568 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.410528 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" event={"ID":"b3052be5-f8ee-4084-a2ca-7b625b79f0f2","Type":"ContainerStarted","Data":"3dac59ab2b1ed8779a07e48500467378054f2e5b64579a14e0456877376a14e7"} Jul 06 19:25:55.469207 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469379 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469230 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469379 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469379 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469379 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.469595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:55.469595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.469469 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99g64\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.472673 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.472636 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f818fd7-c436-4d9a-8d78-ffae230e063a-cert\") pod \"odh-model-controller-7998c67dc4-r5vxj\" (UID: \"7f818fd7-c436-4d9a-8d78-ffae230e063a\") " pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:55.570018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.569919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.569961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570018 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.569983 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570335 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.570040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570335 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.570066 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570335 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.570305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99g64\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.570486 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.570386 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.571055 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.571030 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.572814 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.572781 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.572921 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.572864 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.572985 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.572968 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.573072 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.573052 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.580527 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.580503 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:25:55.587729 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.587699 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99g64\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.589076 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.589046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token\") pod \"istiod-openshift-gateway-7574459656-hrlf5\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.633260 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.633235 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:25:55.728957 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.728922 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r5vxj"] Jul 06 19:25:55.732523 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:55.732493 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f818fd7_c436_4d9a_8d78_ffae230e063a.slice/crio-2f62c34e95a93cd4bbff8cec494097e1d4ae14f5c8a78eb7136125c284eb088d WatchSource:0}: Error finding container 2f62c34e95a93cd4bbff8cec494097e1d4ae14f5c8a78eb7136125c284eb088d: Status 404 returned error can't find the container with id 2f62c34e95a93cd4bbff8cec494097e1d4ae14f5c8a78eb7136125c284eb088d Jul 06 19:25:55.793844 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:55.793810 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:25:55.797024 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:25:55.796891 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273201bb_4562_4529_a3b2_a3b007659fca.slice/crio-6369db1b9e5a66160e39ad22f46d15b0a185420739f131aa06211d155562d613 WatchSource:0}: Error finding container 6369db1b9e5a66160e39ad22f46d15b0a185420739f131aa06211d155562d613: Status 404 returned error can't find the container with id 6369db1b9e5a66160e39ad22f46d15b0a185420739f131aa06211d155562d613 Jul 06 19:25:56.420288 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:56.420249 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" event={"ID":"273201bb-4562-4529-a3b2-a3b007659fca","Type":"ContainerStarted","Data":"6369db1b9e5a66160e39ad22f46d15b0a185420739f131aa06211d155562d613"} Jul 06 19:25:56.422012 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:25:56.421980 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" event={"ID":"7f818fd7-c436-4d9a-8d78-ffae230e063a","Type":"ContainerStarted","Data":"2f62c34e95a93cd4bbff8cec494097e1d4ae14f5c8a78eb7136125c284eb088d"} Jul 06 19:26:00.541758 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:00.541689 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:26:00.542237 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:00.541793 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 19:26:01.458852 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.458814 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" event={"ID":"b3052be5-f8ee-4084-a2ca-7b625b79f0f2","Type":"ContainerStarted","Data":"3e31dd1e482a324b030feb27614a8245c4e1bfa2065162b83638e08b997c0843"} Jul 06 19:26:01.459111 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.458940 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:26:01.460513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.460493 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" event={"ID":"273201bb-4562-4529-a3b2-a3b007659fca","Type":"ContainerStarted","Data":"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6"} Jul 06 19:26:01.460620 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.460593 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:26:01.462169 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.462124 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" event={"ID":"fea0f914-328b-4f48-a9db-65a93c985f1b","Type":"ContainerStarted","Data":"cba1db676c13e08292f03d29072f30d4aba1d47ee0df46d72c77e53bbf64170a"} Jul 06 19:26:01.462272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.462167 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-hrlf5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:26:01.462272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.462213 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" podUID="273201bb-4562-4529-a3b2-a3b007659fca" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:26:01.463606 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.463585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" event={"ID":"7f818fd7-c436-4d9a-8d78-ffae230e063a","Type":"ContainerStarted","Data":"e5008e5a3f15f8acd0a688561c12c4380f53f9b1c3f2e4a4d01b04864b2d3b52"} Jul 06 19:26:01.463721 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.463709 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:26:01.497939 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.497886 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" podStartSLOduration=2.226332832 podStartE2EDuration="7.49787473s" totalImagesPullingTime="5.271541898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:01.49687235 +0000 UTC m=+376.176906148" watchObservedRunningTime="2026-07-06 19:26:01.49787473 +0000 UTC m=+376.177908521" Jul 06 19:26:01.661511 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.661454 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bc75879f5-btxg7" podStartSLOduration=2.216150057 podStartE2EDuration="8.661438464s" totalImagesPullingTime="6.445288407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:01.661365055 +0000 UTC m=+376.341398855" watchObservedRunningTime="2026-07-06 19:26:01.661438464 +0000 UTC m=+376.341472261" Jul 06 19:26:01.663047 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.663017 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" podStartSLOduration=3.785912664 podStartE2EDuration="8.663008926s" totalImagesPullingTime="4.877096262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:01.550285634 +0000 UTC m=+376.230319433" watchObservedRunningTime="2026-07-06 19:26:01.663008926 +0000 UTC m=+376.343042726" Jul 06 19:26:01.778895 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:01.778792 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" podStartSLOduration=2.036570662 podStartE2EDuration="6.778775496s" totalImagesPullingTime="4.742204834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:01.777287985 +0000 UTC m=+376.457321791" watchObservedRunningTime="2026-07-06 19:26:01.778775496 +0000 UTC m=+376.458809293" Jul 06 19:26:02.469619 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:02.469591 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:26:03.941659 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.941625 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz"] Jul 06 19:26:03.944909 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.944892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:03.949341 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949318 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:26:03.949341 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949332 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:26:03.949543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949367 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:26:03.949543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949319 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:26:03.949543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949363 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:26:03.949543 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.949442 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n7cxb\"" Jul 06 19:26:03.964603 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:03.964576 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz"] Jul 06 19:26:04.052917 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.052884 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xknmj\" (UniqueName: \"kubernetes.io/projected/f279d084-267b-4a33-a17e-274bc97545ab-kube-api-access-xknmj\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.053077 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.052928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f279d084-267b-4a33-a17e-274bc97545ab-manager-config\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.053077 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.053006 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.053185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.053108 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-metrics-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.154475 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.154442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-metrics-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.154633 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.154486 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xknmj\" (UniqueName: \"kubernetes.io/projected/f279d084-267b-4a33-a17e-274bc97545ab-kube-api-access-xknmj\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.154633 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.154520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f279d084-267b-4a33-a17e-274bc97545ab-manager-config\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.154633 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.154564 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.155294 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.155268 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f279d084-267b-4a33-a17e-274bc97545ab-manager-config\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.157207 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.157187 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.157268 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.157187 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f279d084-267b-4a33-a17e-274bc97545ab-metrics-cert\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.165158 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.165100 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xknmj\" (UniqueName: \"kubernetes.io/projected/f279d084-267b-4a33-a17e-274bc97545ab-kube-api-access-xknmj\") pod \"lws-controller-manager-7d885dd8f9-t2vmz\" (UID: \"f279d084-267b-4a33-a17e-274bc97545ab\") " pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.254929 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.254841 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:04.414286 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.414259 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz"] Jul 06 19:26:04.415677 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:26:04.415652 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf279d084_267b_4a33_a17e_274bc97545ab.slice/crio-aa9c01027d05e9d0acffd23595b4f8cb008722d468534a73ce2a33ca0acd1cd6 WatchSource:0}: Error finding container aa9c01027d05e9d0acffd23595b4f8cb008722d468534a73ce2a33ca0acd1cd6: Status 404 returned error can't find the container with id aa9c01027d05e9d0acffd23595b4f8cb008722d468534a73ce2a33ca0acd1cd6 Jul 06 19:26:04.479068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:04.479035 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" event={"ID":"f279d084-267b-4a33-a17e-274bc97545ab","Type":"ContainerStarted","Data":"aa9c01027d05e9d0acffd23595b4f8cb008722d468534a73ce2a33ca0acd1cd6"} Jul 06 19:26:06.496011 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:06.495840 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" event={"ID":"f279d084-267b-4a33-a17e-274bc97545ab","Type":"ContainerStarted","Data":"b73b5dc9ba184f1d636fe456a3b98dad8816d1f241c72012e0be12f204265440"} Jul 06 19:26:06.496448 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:06.496234 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:26:06.520200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:06.520152 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" podStartSLOduration=1.58975127 podStartE2EDuration="3.520120988s" totalImagesPullingTime="1.930369718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:06.518328265 +0000 UTC m=+381.198362059" watchObservedRunningTime="2026-07-06 19:26:06.520120988 +0000 UTC m=+381.200154784" Jul 06 19:26:08.623866 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.623831 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:26:08.624262 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.624027 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" podUID="273201bb-4562-4529-a3b2-a3b007659fca" containerName="discovery" containerID="cri-o://ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6" gracePeriod=30 Jul 06 19:26:08.880095 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.880032 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:26:08.891885 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.891856 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892014 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.891906 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892014 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.891937 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892014 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.891967 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99g64\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892193 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.892033 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892193 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.892096 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.892193 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.892143 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert\") pod \"273201bb-4562-4529-a3b2-a3b007659fca\" (UID: \"273201bb-4562-4529-a3b2-a3b007659fca\") " Jul 06 19:26:08.893052 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.892605 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:08.894708 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.894663 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token" (OuterVolumeSpecName: "istio-token") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:08.895041 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.895017 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs" (OuterVolumeSpecName: "local-certs") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:26:08.895796 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.895741 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:08.895796 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.895763 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64" (OuterVolumeSpecName: "kube-api-access-99g64") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "kube-api-access-99g64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:08.895905 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.895824 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts" (OuterVolumeSpecName: "cacerts") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:08.896415 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.896393 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "273201bb-4562-4529-a3b2-a3b007659fca" (UID: "273201bb-4562-4529-a3b2-a3b007659fca"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:08.993684 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993645 2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-cacerts\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993684 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993683 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-dns-cert\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993697 2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/273201bb-4562-4529-a3b2-a3b007659fca-istio-kubeconfig\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993710 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-istio-token\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993723 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/273201bb-4562-4529-a3b2-a3b007659fca-istio-csr-ca-configmap\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993735 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99g64\" (UniqueName: \"kubernetes.io/projected/273201bb-4562-4529-a3b2-a3b007659fca-kube-api-access-99g64\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:08.993874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:08.993747 2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/273201bb-4562-4529-a3b2-a3b007659fca-local-certs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:26:09.510936 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.510896 2576 generic.go:356] "Generic (PLEG): container finished" podID="273201bb-4562-4529-a3b2-a3b007659fca" containerID="ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6" exitCode=0 Jul 06 19:26:09.511085 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.510967 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" Jul 06 19:26:09.511085 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.510985 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" event={"ID":"273201bb-4562-4529-a3b2-a3b007659fca","Type":"ContainerDied","Data":"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6"} Jul 06 19:26:09.511085 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.511067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5" event={"ID":"273201bb-4562-4529-a3b2-a3b007659fca","Type":"ContainerDied","Data":"6369db1b9e5a66160e39ad22f46d15b0a185420739f131aa06211d155562d613"} Jul 06 19:26:09.511272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.511091 2576 scope.go:117] "RemoveContainer" containerID="ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6" Jul 06 19:26:09.519946 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.519927 2576 scope.go:117] "RemoveContainer" containerID="ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6" Jul 06 19:26:09.520232 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:26:09.520212 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6\": container with ID starting with ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6 not found: ID does not exist" containerID="ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6" Jul 06 19:26:09.520316 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.520238 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6"} err="failed to get container status \"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6\": rpc error: code = NotFound desc = could not find container \"ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6\": container with ID starting with ee2fb56c977831164ba021230309266a677a8a40591ac4808ed584875b1495d6 not found: ID does not exist" Jul 06 19:26:09.542843 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.542818 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:26:09.571426 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.571401 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-hrlf5"] Jul 06 19:26:09.849521 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:09.849447 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="273201bb-4562-4529-a3b2-a3b007659fca" path="/var/lib/kubelet/pods/273201bb-4562-4529-a3b2-a3b007659fca/volumes" Jul 06 19:26:12.471736 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:12.471708 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r5vxj" Jul 06 19:26:12.473460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:12.473441 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gqb7x" Jul 06 19:26:17.503436 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:26:17.503404 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d885dd8f9-t2vmz" Jul 06 19:27:10.238059 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.237981 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl"] Jul 06 19:27:10.238512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.238328 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273201bb-4562-4529-a3b2-a3b007659fca" containerName="discovery" Jul 06 19:27:10.238512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.238341 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="273201bb-4562-4529-a3b2-a3b007659fca" containerName="discovery" Jul 06 19:27:10.238512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.238419 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="273201bb-4562-4529-a3b2-a3b007659fca" containerName="discovery" Jul 06 19:27:10.240368 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.240348 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.242882 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.242862 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:27:10.243364 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.243339 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:27:10.244052 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.244034 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6mgqb\"" Jul 06 19:27:10.250639 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.250617 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl"] Jul 06 19:27:10.292549 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.292523 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9mcs\" (UniqueName: \"kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.292673 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.292578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.292720 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.292676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.393963 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.393926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.394117 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.393994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.394117 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.394015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9mcs\" (UniqueName: \"kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.394321 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.394299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.394385 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.394359 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.403431 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.403404 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9mcs\" (UniqueName: \"kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.551046 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.550968 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:10.677467 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.677441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl"] Jul 06 19:27:10.679854 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:27:10.679830 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c9ef81_648c_428f_8e8e_f5e21f36ac27.slice/crio-25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02 WatchSource:0}: Error finding container 25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02: Status 404 returned error can't find the container with id 25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02 Jul 06 19:27:10.788293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.788269 2576 generic.go:356] "Generic (PLEG): container finished" podID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerID="d39650a31d10b26759c1ca9efd2974f1b2db74f3a9c4e9bafd2bfba16a62259d" exitCode=0 Jul 06 19:27:10.788389 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.788312 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" event={"ID":"55c9ef81-648c-428f-8e8e-f5e21f36ac27","Type":"ContainerDied","Data":"d39650a31d10b26759c1ca9efd2974f1b2db74f3a9c4e9bafd2bfba16a62259d"} Jul 06 19:27:10.788389 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:10.788348 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" event={"ID":"55c9ef81-648c-428f-8e8e-f5e21f36ac27","Type":"ContainerStarted","Data":"25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02"} Jul 06 19:27:11.037925 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.034982 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc"] Jul 06 19:27:11.039059 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.039032 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.055091 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.055066 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc"] Jul 06 19:27:11.101258 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.101232 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.101350 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.101262 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8jts\" (UniqueName: \"kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.101350 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.101332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.201797 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.201754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.201914 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.201829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.201914 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.201852 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8jts\" (UniqueName: \"kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.202177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.202158 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.202252 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.202185 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.211617 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.211596 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8jts\" (UniqueName: \"kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.348367 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.348308 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:11.496095 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.496063 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc"] Jul 06 19:27:11.525746 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:27:11.525717 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76bff87b_1ee9_4f47_a743_c75ff7f2c2b3.slice/crio-3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a WatchSource:0}: Error finding container 3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a: Status 404 returned error can't find the container with id 3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a Jul 06 19:27:11.643911 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.643882 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl"] Jul 06 19:27:11.646718 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.646698 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.655968 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.655947 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl"] Jul 06 19:27:11.706191 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.706116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.706305 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.706246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnj9h\" (UniqueName: \"kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.706357 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.706344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.794658 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.794634 2576 generic.go:356] "Generic (PLEG): container finished" podID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerID="2030c9a1912a5dc7bc8bf68a2021bcf4febfedacd21e33149961adff8a019b9b" exitCode=0 Jul 06 19:27:11.794771 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.794724 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" event={"ID":"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3","Type":"ContainerDied","Data":"2030c9a1912a5dc7bc8bf68a2021bcf4febfedacd21e33149961adff8a019b9b"} Jul 06 19:27:11.794826 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.794786 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" event={"ID":"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3","Type":"ContainerStarted","Data":"3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a"} Jul 06 19:27:11.796592 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.796572 2576 generic.go:356] "Generic (PLEG): container finished" podID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerID="beb617c4e37232556d414f2f809a0b95d82f0efdebf348719ef94f9285c9a54e" exitCode=0 Jul 06 19:27:11.796668 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.796603 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" event={"ID":"55c9ef81-648c-428f-8e8e-f5e21f36ac27","Type":"ContainerDied","Data":"beb617c4e37232556d414f2f809a0b95d82f0efdebf348719ef94f9285c9a54e"} Jul 06 19:27:11.807666 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.807646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.807758 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.807689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnj9h\" (UniqueName: \"kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.807845 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.807756 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.808061 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.808039 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.808165 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.808063 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:11.820838 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:11.820813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnj9h\" (UniqueName: \"kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:12.036577 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.036550 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2"] Jul 06 19:27:12.039631 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.039612 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.048022 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.047996 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2"] Jul 06 19:27:12.066275 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.066251 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:12.110951 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.110922 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.111089 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.110975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkt49\" (UniqueName: \"kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.111089 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.111026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.192907 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.192877 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl"] Jul 06 19:27:12.195853 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:27:12.195826 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d017f9_9690_4308_8bdb_dd20171413e9.slice/crio-62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30 WatchSource:0}: Error finding container 62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30: Status 404 returned error can't find the container with id 62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30 Jul 06 19:27:12.211901 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.211882 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.211969 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.211929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkt49\" (UniqueName: \"kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.211969 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.211960 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.212293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.212274 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.212338 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.212279 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.220062 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.220039 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkt49\" (UniqueName: \"kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.351737 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.351706 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:12.481008 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.480985 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2"] Jul 06 19:27:12.483098 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:27:12.483070 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2e2c28_043f_48a0_8c64_d3a28efc960a.slice/crio-5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc WatchSource:0}: Error finding container 5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc: Status 404 returned error can't find the container with id 5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc Jul 06 19:27:12.802904 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.802824 2576 generic.go:356] "Generic (PLEG): container finished" podID="99d017f9-9690-4308-8bdb-dd20171413e9" containerID="dfec43a4b653084c580644b3d10fc390e72abd96db84387b0d1569db775993da" exitCode=0 Jul 06 19:27:12.802904 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.802886 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" event={"ID":"99d017f9-9690-4308-8bdb-dd20171413e9","Type":"ContainerDied","Data":"dfec43a4b653084c580644b3d10fc390e72abd96db84387b0d1569db775993da"} Jul 06 19:27:12.803106 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.802942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" event={"ID":"99d017f9-9690-4308-8bdb-dd20171413e9","Type":"ContainerStarted","Data":"62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30"} Jul 06 19:27:12.804826 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.804805 2576 generic.go:356] "Generic (PLEG): container finished" podID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerID="c104b3726c897d0f9a1b96c97935936433e0d62bf50627e9959ce521a31a3b82" exitCode=0 Jul 06 19:27:12.804987 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.804877 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" event={"ID":"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3","Type":"ContainerDied","Data":"c104b3726c897d0f9a1b96c97935936433e0d62bf50627e9959ce521a31a3b82"} Jul 06 19:27:12.807030 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.806998 2576 generic.go:356] "Generic (PLEG): container finished" podID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerID="345dd256ea5a7e1803fa643fbccdd52d9900d5e87bcc09159841144f89592d4e" exitCode=0 Jul 06 19:27:12.807120 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.807023 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" event={"ID":"55c9ef81-648c-428f-8e8e-f5e21f36ac27","Type":"ContainerDied","Data":"345dd256ea5a7e1803fa643fbccdd52d9900d5e87bcc09159841144f89592d4e"} Jul 06 19:27:12.808556 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.808536 2576 generic.go:356] "Generic (PLEG): container finished" podID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerID="e9935962c55ff5e68f7cb6ce773c33236c0440452ef845278ce7cb0f5c6e0fc0" exitCode=0 Jul 06 19:27:12.808650 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.808624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" event={"ID":"8b2e2c28-043f-48a0-8c64-d3a28efc960a","Type":"ContainerDied","Data":"e9935962c55ff5e68f7cb6ce773c33236c0440452ef845278ce7cb0f5c6e0fc0"} Jul 06 19:27:12.808714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:12.808667 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" event={"ID":"8b2e2c28-043f-48a0-8c64-d3a28efc960a","Type":"ContainerStarted","Data":"5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc"} Jul 06 19:27:13.815498 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.815464 2576 generic.go:356] "Generic (PLEG): container finished" podID="99d017f9-9690-4308-8bdb-dd20171413e9" containerID="0698d43018ff4e98daf41072936ecb745eec528461f2b19766d9cd167a171fb7" exitCode=0 Jul 06 19:27:13.815886 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.815555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" event={"ID":"99d017f9-9690-4308-8bdb-dd20171413e9","Type":"ContainerDied","Data":"0698d43018ff4e98daf41072936ecb745eec528461f2b19766d9cd167a171fb7"} Jul 06 19:27:13.817896 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.817872 2576 generic.go:356] "Generic (PLEG): container finished" podID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerID="5cba694a5b71710b6e44aba65b0fd72313c17382cf9632f3566cf9abba8fbd70" exitCode=0 Jul 06 19:27:13.817990 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.817902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" event={"ID":"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3","Type":"ContainerDied","Data":"5cba694a5b71710b6e44aba65b0fd72313c17382cf9632f3566cf9abba8fbd70"} Jul 06 19:27:13.819891 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.819870 2576 generic.go:356] "Generic (PLEG): container finished" podID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerID="1536f8fe21114da35de74c57ff691b423bc4b9bf7fd49f70c60d2644ae6ee111" exitCode=0 Jul 06 19:27:13.819976 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.819955 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" event={"ID":"8b2e2c28-043f-48a0-8c64-d3a28efc960a","Type":"ContainerDied","Data":"1536f8fe21114da35de74c57ff691b423bc4b9bf7fd49f70c60d2644ae6ee111"} Jul 06 19:27:13.951589 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:13.951565 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:14.029482 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.029454 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle\") pod \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " Jul 06 19:27:14.029647 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.029504 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9mcs\" (UniqueName: \"kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs\") pod \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " Jul 06 19:27:14.029647 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.029522 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util\") pod \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\" (UID: \"55c9ef81-648c-428f-8e8e-f5e21f36ac27\") " Jul 06 19:27:14.029986 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.029960 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle" (OuterVolumeSpecName: "bundle") pod "55c9ef81-648c-428f-8e8e-f5e21f36ac27" (UID: "55c9ef81-648c-428f-8e8e-f5e21f36ac27"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:14.031646 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.031625 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs" (OuterVolumeSpecName: "kube-api-access-j9mcs") pod "55c9ef81-648c-428f-8e8e-f5e21f36ac27" (UID: "55c9ef81-648c-428f-8e8e-f5e21f36ac27"). InnerVolumeSpecName "kube-api-access-j9mcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:14.035122 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.035100 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util" (OuterVolumeSpecName: "util") pod "55c9ef81-648c-428f-8e8e-f5e21f36ac27" (UID: "55c9ef81-648c-428f-8e8e-f5e21f36ac27"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:14.130750 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.130671 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:14.130750 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.130705 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9mcs\" (UniqueName: \"kubernetes.io/projected/55c9ef81-648c-428f-8e8e-f5e21f36ac27-kube-api-access-j9mcs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:14.130750 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.130716 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55c9ef81-648c-428f-8e8e-f5e21f36ac27-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:14.827040 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.827009 2576 generic.go:356] "Generic (PLEG): container finished" podID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerID="23fccf986e22b3ba9978522cc5a6f47146cca330372b63045e9595740f0e203c" exitCode=0 Jul 06 19:27:14.827495 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.827088 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" event={"ID":"8b2e2c28-043f-48a0-8c64-d3a28efc960a","Type":"ContainerDied","Data":"23fccf986e22b3ba9978522cc5a6f47146cca330372b63045e9595740f0e203c"} Jul 06 19:27:14.829005 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.828979 2576 generic.go:356] "Generic (PLEG): container finished" podID="99d017f9-9690-4308-8bdb-dd20171413e9" containerID="2e9eccf5d060afd58d736f34c5c71e95473738434009fcb4e907174a6feb8743" exitCode=0 Jul 06 19:27:14.829127 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.829062 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" event={"ID":"99d017f9-9690-4308-8bdb-dd20171413e9","Type":"ContainerDied","Data":"2e9eccf5d060afd58d736f34c5c71e95473738434009fcb4e907174a6feb8743"} Jul 06 19:27:14.830851 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.830831 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" Jul 06 19:27:14.830938 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.830865 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl" event={"ID":"55c9ef81-648c-428f-8e8e-f5e21f36ac27","Type":"ContainerDied","Data":"25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02"} Jul 06 19:27:14.830938 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.830904 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25821c721b585e91b6b78d2b3736b5926e44f256edd436df8a4343df0ae7bf02" Jul 06 19:27:14.960352 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:14.960331 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:15.037813 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.037788 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util\") pod \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " Jul 06 19:27:15.037988 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.037844 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle\") pod \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " Jul 06 19:27:15.037988 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.037883 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8jts\" (UniqueName: \"kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts\") pod \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\" (UID: \"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3\") " Jul 06 19:27:15.038475 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.038446 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle" (OuterVolumeSpecName: "bundle") pod "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" (UID: "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:15.039969 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.039940 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts" (OuterVolumeSpecName: "kube-api-access-d8jts") pod "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" (UID: "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3"). InnerVolumeSpecName "kube-api-access-d8jts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:15.043214 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.043181 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util" (OuterVolumeSpecName: "util") pod "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" (UID: "76bff87b-1ee9-4f47-a743-c75ff7f2c2b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:15.138637 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.138566 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:15.138637 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.138592 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:15.138637 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.138615 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8jts\" (UniqueName: \"kubernetes.io/projected/76bff87b-1ee9-4f47-a743-c75ff7f2c2b3-kube-api-access-d8jts\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:15.838668 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.838644 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" Jul 06 19:27:15.839053 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.838678 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc" event={"ID":"76bff87b-1ee9-4f47-a743-c75ff7f2c2b3","Type":"ContainerDied","Data":"3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a"} Jul 06 19:27:15.839053 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.838735 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d0dcbbf6a704b6bcc3685e20784348a289d380e482d41253956161c1091165a" Jul 06 19:27:15.986447 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:15.986425 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:16.014263 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.014243 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:16.046785 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046761 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle\") pod \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " Jul 06 19:27:16.046912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046802 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util\") pod \"99d017f9-9690-4308-8bdb-dd20171413e9\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " Jul 06 19:27:16.046912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046853 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkt49\" (UniqueName: \"kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49\") pod \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " Jul 06 19:27:16.046912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046875 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnj9h\" (UniqueName: \"kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h\") pod \"99d017f9-9690-4308-8bdb-dd20171413e9\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " Jul 06 19:27:16.046912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046898 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle\") pod \"99d017f9-9690-4308-8bdb-dd20171413e9\" (UID: \"99d017f9-9690-4308-8bdb-dd20171413e9\") " Jul 06 19:27:16.047120 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.046924 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util\") pod \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\" (UID: \"8b2e2c28-043f-48a0-8c64-d3a28efc960a\") " Jul 06 19:27:16.047595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.047565 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle" (OuterVolumeSpecName: "bundle") pod "99d017f9-9690-4308-8bdb-dd20171413e9" (UID: "99d017f9-9690-4308-8bdb-dd20171413e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:16.047685 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.047574 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle" (OuterVolumeSpecName: "bundle") pod "8b2e2c28-043f-48a0-8c64-d3a28efc960a" (UID: "8b2e2c28-043f-48a0-8c64-d3a28efc960a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:16.049439 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.049414 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h" (OuterVolumeSpecName: "kube-api-access-lnj9h") pod "99d017f9-9690-4308-8bdb-dd20171413e9" (UID: "99d017f9-9690-4308-8bdb-dd20171413e9"). InnerVolumeSpecName "kube-api-access-lnj9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:16.049579 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.049517 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49" (OuterVolumeSpecName: "kube-api-access-jkt49") pod "8b2e2c28-043f-48a0-8c64-d3a28efc960a" (UID: "8b2e2c28-043f-48a0-8c64-d3a28efc960a"). InnerVolumeSpecName "kube-api-access-jkt49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:16.053088 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.053062 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util" (OuterVolumeSpecName: "util") pod "8b2e2c28-043f-48a0-8c64-d3a28efc960a" (UID: "8b2e2c28-043f-48a0-8c64-d3a28efc960a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:16.053355 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.053335 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util" (OuterVolumeSpecName: "util") pod "99d017f9-9690-4308-8bdb-dd20171413e9" (UID: "99d017f9-9690-4308-8bdb-dd20171413e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148462 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148487 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b2e2c28-043f-48a0-8c64-d3a28efc960a-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148496 2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-util\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148504 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkt49\" (UniqueName: \"kubernetes.io/projected/8b2e2c28-043f-48a0-8c64-d3a28efc960a-kube-api-access-jkt49\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148514 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnj9h\" (UniqueName: \"kubernetes.io/projected/99d017f9-9690-4308-8bdb-dd20171413e9-kube-api-access-lnj9h\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.148530 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.148523 2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d017f9-9690-4308-8bdb-dd20171413e9-bundle\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:27:16.844564 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.844540 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" Jul 06 19:27:16.844997 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.844582 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2" event={"ID":"8b2e2c28-043f-48a0-8c64-d3a28efc960a","Type":"ContainerDied","Data":"5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc"} Jul 06 19:27:16.844997 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.844619 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b686a82c5ba88e9423f54f9e874e6d3aabd559431d00ba5d9acf03f2bd8b5dc" Jul 06 19:27:16.846519 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.846499 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" Jul 06 19:27:16.846519 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.846504 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl" event={"ID":"99d017f9-9690-4308-8bdb-dd20171413e9","Type":"ContainerDied","Data":"62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30"} Jul 06 19:27:16.846637 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:27:16.846540 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62258bc1836493bea226902a70c2fdf6d720b9d2b792324a017723329fafcd30" Jul 06 19:28:28.507385 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507343 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507827 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="util" Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507847 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="util" Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507858 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="extract" Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507867 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="extract" Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507877 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="extract" Jul 06 19:28:28.507887 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507886 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507898 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507907 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507916 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507924 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507933 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507941 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507950 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507958 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507966 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507973 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507985 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.507993 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508007 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508015 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508022 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508030 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="util" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508049 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508057 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="pull" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508160 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55c9ef81-648c-428f-8e8e-f5e21f36ac27" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508172 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b2e2c28-043f-48a0-8c64-d3a28efc960a" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508186 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d017f9-9690-4308-8bdb-dd20171413e9" containerName="extract" Jul 06 19:28:28.508238 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.508197 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="76bff87b-1ee9-4f47-a743-c75ff7f2c2b3" containerName="extract" Jul 06 19:28:28.511455 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.511434 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.514023 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.513999 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 19:28:28.514265 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.514247 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:28:28.515112 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.515098 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6mgqb\"" Jul 06 19:28:28.515237 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.515218 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:28:28.528412 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.528391 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:28:28.573745 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.573711 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:28:28.619006 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.618970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-config-file\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.619218 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.619051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2snrc\" (UniqueName: \"kubernetes.io/projected/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-kube-api-access-2snrc\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.719945 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.719910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-config-file\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.720128 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.719970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2snrc\" (UniqueName: \"kubernetes.io/projected/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-kube-api-access-2snrc\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.720667 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.720631 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-config-file\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.732107 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.732086 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2snrc\" (UniqueName: \"kubernetes.io/projected/0494c6aa-acfe-4fd0-8752-5d144bcb30c7-kube-api-access-2snrc\") pod \"limitador-limitador-54d6d988d5-s8s82\" (UID: \"0494c6aa-acfe-4fd0-8752-5d144bcb30c7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.821831 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.821765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:28.953684 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:28.953648 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:28:28.955488 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:28:28.955460 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0494c6aa_acfe_4fd0_8752_5d144bcb30c7.slice/crio-1285ff89e2a1705df54caba6b04a9ead6d309d4105973aa1846dcec42e1cd6cf WatchSource:0}: Error finding container 1285ff89e2a1705df54caba6b04a9ead6d309d4105973aa1846dcec42e1cd6cf: Status 404 returned error can't find the container with id 1285ff89e2a1705df54caba6b04a9ead6d309d4105973aa1846dcec42e1cd6cf Jul 06 19:28:29.063670 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.063642 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:29.068068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.068041 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:29.070541 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.070522 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m7fq6\"" Jul 06 19:28:29.075172 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.075085 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:29.122565 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.122535 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67g77\" (UniqueName: \"kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77\") pod \"authorino-58cd955675-k9fxm\" (UID: \"b71de4ca-e49d-4146-9731-a6f65e588d92\") " pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:29.184159 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.184102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" event={"ID":"0494c6aa-acfe-4fd0-8752-5d144bcb30c7","Type":"ContainerStarted","Data":"1285ff89e2a1705df54caba6b04a9ead6d309d4105973aa1846dcec42e1cd6cf"} Jul 06 19:28:29.223325 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.223292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67g77\" (UniqueName: \"kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77\") pod \"authorino-58cd955675-k9fxm\" (UID: \"b71de4ca-e49d-4146-9731-a6f65e588d92\") " pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:29.231489 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.231463 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67g77\" (UniqueName: \"kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77\") pod \"authorino-58cd955675-k9fxm\" (UID: \"b71de4ca-e49d-4146-9731-a6f65e588d92\") " pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:29.377961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.377806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:29.537934 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:29.537903 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:29.538875 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:28:29.538844 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb71de4ca_e49d_4146_9731_a6f65e588d92.slice/crio-61d7df853a9d2033c5bb3259636f8c0a1841c212e6274da5a6240a0931188958 WatchSource:0}: Error finding container 61d7df853a9d2033c5bb3259636f8c0a1841c212e6274da5a6240a0931188958: Status 404 returned error can't find the container with id 61d7df853a9d2033c5bb3259636f8c0a1841c212e6274da5a6240a0931188958 Jul 06 19:28:30.191070 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:30.191028 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k9fxm" event={"ID":"b71de4ca-e49d-4146-9731-a6f65e588d92","Type":"ContainerStarted","Data":"61d7df853a9d2033c5bb3259636f8c0a1841c212e6274da5a6240a0931188958"} Jul 06 19:28:32.205941 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:32.205890 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" event={"ID":"0494c6aa-acfe-4fd0-8752-5d144bcb30c7","Type":"ContainerStarted","Data":"30e374e62de4dea9c1e8f50969f94d481ae3f4f443e57c94eadff740f6e2805e"} Jul 06 19:28:32.206472 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:32.205993 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:28:32.229153 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:32.229055 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" podStartSLOduration=1.686201809 podStartE2EDuration="4.229037013s" totalImagesPullingTime="2.542835204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:32.22691387 +0000 UTC m=+526.906947679" watchObservedRunningTime="2026-07-06 19:28:32.229037013 +0000 UTC m=+526.909070811" Jul 06 19:28:34.217366 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:34.217334 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k9fxm" event={"ID":"b71de4ca-e49d-4146-9731-a6f65e588d92","Type":"ContainerStarted","Data":"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe"} Jul 06 19:28:34.238098 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:34.238049 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k9fxm" podStartSLOduration=0.895095255 podStartE2EDuration="5.238035917s" totalImagesPullingTime="4.342940662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:34.235488117 +0000 UTC m=+528.915521913" watchObservedRunningTime="2026-07-06 19:28:34.238035917 +0000 UTC m=+528.918069717" Jul 06 19:28:35.165929 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:35.165897 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:36.227252 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:36.227215 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k9fxm" podUID="b71de4ca-e49d-4146-9731-a6f65e588d92" containerName="authorino" containerID="cri-o://22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe" gracePeriod=30 Jul 06 19:28:36.476174 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:36.476151 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:36.583333 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:36.583262 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67g77\" (UniqueName: \"kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77\") pod \"b71de4ca-e49d-4146-9731-a6f65e588d92\" (UID: \"b71de4ca-e49d-4146-9731-a6f65e588d92\") " Jul 06 19:28:36.585208 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:36.585184 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77" (OuterVolumeSpecName: "kube-api-access-67g77") pod "b71de4ca-e49d-4146-9731-a6f65e588d92" (UID: "b71de4ca-e49d-4146-9731-a6f65e588d92"). InnerVolumeSpecName "kube-api-access-67g77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:28:36.684383 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:36.684356 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67g77\" (UniqueName: \"kubernetes.io/projected/b71de4ca-e49d-4146-9731-a6f65e588d92-kube-api-access-67g77\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:28:37.233345 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.233309 2576 generic.go:356] "Generic (PLEG): container finished" podID="b71de4ca-e49d-4146-9731-a6f65e588d92" containerID="22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe" exitCode=0 Jul 06 19:28:37.233735 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.233352 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k9fxm" Jul 06 19:28:37.233735 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.233395 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k9fxm" event={"ID":"b71de4ca-e49d-4146-9731-a6f65e588d92","Type":"ContainerDied","Data":"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe"} Jul 06 19:28:37.233735 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.233439 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k9fxm" event={"ID":"b71de4ca-e49d-4146-9731-a6f65e588d92","Type":"ContainerDied","Data":"61d7df853a9d2033c5bb3259636f8c0a1841c212e6274da5a6240a0931188958"} Jul 06 19:28:37.233735 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.233460 2576 scope.go:117] "RemoveContainer" containerID="22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe" Jul 06 19:28:37.242230 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.242209 2576 scope.go:117] "RemoveContainer" containerID="22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe" Jul 06 19:28:37.242477 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:28:37.242458 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe\": container with ID starting with 22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe not found: ID does not exist" containerID="22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe" Jul 06 19:28:37.242547 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.242485 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe"} err="failed to get container status \"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe\": rpc error: code = NotFound desc = could not find container \"22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe\": container with ID starting with 22de1bc5d7f8a8834a8cec3a7120d333271d4fc8df4d66f77e1f8cbf09f39afe not found: ID does not exist" Jul 06 19:28:37.254960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.254932 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:37.261514 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.261495 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k9fxm"] Jul 06 19:28:37.849433 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:37.849404 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b71de4ca-e49d-4146-9731-a6f65e588d92" path="/var/lib/kubelet/pods/b71de4ca-e49d-4146-9731-a6f65e588d92/volumes" Jul 06 19:28:43.212191 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:28:43.212159 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-s8s82" Jul 06 19:29:01.822389 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.822353 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ggngb"] Jul 06 19:29:01.822855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.822688 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b71de4ca-e49d-4146-9731-a6f65e588d92" containerName="authorino" Jul 06 19:29:01.822855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.822699 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b71de4ca-e49d-4146-9731-a6f65e588d92" containerName="authorino" Jul 06 19:29:01.822855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.822757 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b71de4ca-e49d-4146-9731-a6f65e588d92" containerName="authorino" Jul 06 19:29:01.828849 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.828826 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:01.832347 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.832323 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m7fq6\"" Jul 06 19:29:01.836005 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.835983 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ggngb"] Jul 06 19:29:01.992295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:01.992256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwh9\" (UniqueName: \"kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9\") pod \"authorino-5d4b67c446-ggngb\" (UID: \"403333bf-e5ff-4874-a35d-49d9678465d8\") " pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:02.044570 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.044530 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ggngb"] Jul 06 19:29:02.044801 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:29:02.044782 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ppwh9], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-ggngb" podUID="403333bf-e5ff-4874-a35d-49d9678465d8" Jul 06 19:29:02.073749 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.073673 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cd5c46748-bx8bm"] Jul 06 19:29:02.075833 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.075817 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.084702 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.084671 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cd5c46748-bx8bm"] Jul 06 19:29:02.092860 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.092840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwh9\" (UniqueName: \"kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9\") pod \"authorino-5d4b67c446-ggngb\" (UID: \"403333bf-e5ff-4874-a35d-49d9678465d8\") " pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:02.103095 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.103068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwh9\" (UniqueName: \"kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9\") pod \"authorino-5d4b67c446-ggngb\" (UID: \"403333bf-e5ff-4874-a35d-49d9678465d8\") " pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:02.193944 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.193916 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhn5m\" (UniqueName: \"kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m\") pod \"authorino-5cd5c46748-bx8bm\" (UID: \"0831dbba-d97c-459d-879d-397cbb1d974a\") " pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.250493 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.250462 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cd5c46748-bx8bm"] Jul 06 19:29:02.250710 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:29:02.250686 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hhn5m], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5cd5c46748-bx8bm" podUID="0831dbba-d97c-459d-879d-397cbb1d974a" Jul 06 19:29:02.286465 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.286439 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:29:02.288911 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.288895 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.291707 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.291690 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 19:29:02.295172 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.295154 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhn5m\" (UniqueName: \"kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m\") pod \"authorino-5cd5c46748-bx8bm\" (UID: \"0831dbba-d97c-459d-879d-397cbb1d974a\") " pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.300403 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.300382 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:29:02.304528 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.304496 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhn5m\" (UniqueName: \"kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m\") pod \"authorino-5cd5c46748-bx8bm\" (UID: \"0831dbba-d97c-459d-879d-397cbb1d974a\") " pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.354210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.354121 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.354322 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.354141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:02.359837 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.359819 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:02.362630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.362614 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:02.396276 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.396254 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.396389 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.396312 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwqp7\" (UniqueName: \"kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.497297 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.497271 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppwh9\" (UniqueName: \"kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9\") pod \"403333bf-e5ff-4874-a35d-49d9678465d8\" (UID: \"403333bf-e5ff-4874-a35d-49d9678465d8\") " Jul 06 19:29:02.497422 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.497353 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhn5m\" (UniqueName: \"kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m\") pod \"0831dbba-d97c-459d-879d-397cbb1d974a\" (UID: \"0831dbba-d97c-459d-879d-397cbb1d974a\") " Jul 06 19:29:02.497531 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.497515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwqp7\" (UniqueName: \"kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.497609 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.497596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.499409 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.499383 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9" (OuterVolumeSpecName: "kube-api-access-ppwh9") pod "403333bf-e5ff-4874-a35d-49d9678465d8" (UID: "403333bf-e5ff-4874-a35d-49d9678465d8"). InnerVolumeSpecName "kube-api-access-ppwh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:02.499474 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.499421 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m" (OuterVolumeSpecName: "kube-api-access-hhn5m") pod "0831dbba-d97c-459d-879d-397cbb1d974a" (UID: "0831dbba-d97c-459d-879d-397cbb1d974a"). InnerVolumeSpecName "kube-api-access-hhn5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:02.499908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.499890 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.506522 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.506493 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwqp7\" (UniqueName: \"kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7\") pod \"authorino-5fb5cddc88-7r42j\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.598253 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.598214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:29:02.598444 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.598396 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppwh9\" (UniqueName: \"kubernetes.io/projected/403333bf-e5ff-4874-a35d-49d9678465d8-kube-api-access-ppwh9\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:29:02.598444 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.598414 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhn5m\" (UniqueName: \"kubernetes.io/projected/0831dbba-d97c-459d-879d-397cbb1d974a-kube-api-access-hhn5m\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:29:02.731277 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:02.731255 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:29:02.732703 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:29:02.732675 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2df08284_6bef_495d_a7b0_7d5758025902.slice/crio-419989d75ff9fb739fe40e7188e15ad98158da099f331ac34cefcef8bf663874 WatchSource:0}: Error finding container 419989d75ff9fb739fe40e7188e15ad98158da099f331ac34cefcef8bf663874: Status 404 returned error can't find the container with id 419989d75ff9fb739fe40e7188e15ad98158da099f331ac34cefcef8bf663874 Jul 06 19:29:03.360272 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.360233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" event={"ID":"2df08284-6bef-495d-a7b0-7d5758025902","Type":"ContainerStarted","Data":"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6"} Jul 06 19:29:03.360714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.360283 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" event={"ID":"2df08284-6bef-495d-a7b0-7d5758025902","Type":"ContainerStarted","Data":"419989d75ff9fb739fe40e7188e15ad98158da099f331ac34cefcef8bf663874"} Jul 06 19:29:03.360714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.360290 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ggngb" Jul 06 19:29:03.360714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.360455 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cd5c46748-bx8bm" Jul 06 19:29:03.381289 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.381210 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" podStartSLOduration=0.989213913 podStartE2EDuration="1.381194848s" totalImagesPullingTime="391.980935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:03.38023602 +0000 UTC m=+558.060269829" watchObservedRunningTime="2026-07-06 19:29:03.381194848 +0000 UTC m=+558.061228690" Jul 06 19:29:03.413058 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.413024 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ggngb"] Jul 06 19:29:03.429477 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.429450 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ggngb"] Jul 06 19:29:03.470485 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.470446 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cd5c46748-bx8bm"] Jul 06 19:29:03.474415 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.474392 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cd5c46748-bx8bm"] Jul 06 19:29:03.849763 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.849729 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0831dbba-d97c-459d-879d-397cbb1d974a" path="/var/lib/kubelet/pods/0831dbba-d97c-459d-879d-397cbb1d974a/volumes" Jul 06 19:29:03.850023 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:03.850006 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403333bf-e5ff-4874-a35d-49d9678465d8" path="/var/lib/kubelet/pods/403333bf-e5ff-4874-a35d-49d9678465d8/volumes" Jul 06 19:29:45.712690 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:45.712663 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:29:45.713212 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:29:45.712922 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:30:09.997234 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:30:09.997148 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:30:34.418325 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:30:34.418287 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:30:44.343467 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:30:44.343433 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:30:56.217533 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:30:56.217491 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:31:07.933525 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:31:07.933492 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:31:32.888475 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:31:32.888436 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:31:42.654311 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:31:42.654236 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:32:13.701596 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.701560 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-788c7f4f8f-6sxcn"] Jul 06 19:32:13.704700 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.704683 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.715624 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.715597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-788c7f4f8f-6sxcn"] Jul 06 19:32:13.733381 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.733353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrhv\" (UniqueName: \"kubernetes.io/projected/c20c57bc-8267-4f52-8608-f2e773a1abc1-kube-api-access-6wrhv\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.733531 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.733407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c20c57bc-8267-4f52-8608-f2e773a1abc1-tls-cert\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.833835 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.833795 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c20c57bc-8267-4f52-8608-f2e773a1abc1-tls-cert\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.834019 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.833860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrhv\" (UniqueName: \"kubernetes.io/projected/c20c57bc-8267-4f52-8608-f2e773a1abc1-kube-api-access-6wrhv\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.836380 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.836355 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c20c57bc-8267-4f52-8608-f2e773a1abc1-tls-cert\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:13.841965 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:13.841938 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrhv\" (UniqueName: \"kubernetes.io/projected/c20c57bc-8267-4f52-8608-f2e773a1abc1-kube-api-access-6wrhv\") pod \"authorino-788c7f4f8f-6sxcn\" (UID: \"c20c57bc-8267-4f52-8608-f2e773a1abc1\") " pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:14.014378 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:14.014311 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" Jul 06 19:32:14.151270 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:14.151245 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-788c7f4f8f-6sxcn"] Jul 06 19:32:14.152778 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:32:14.152750 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc20c57bc_8267_4f52_8608_f2e773a1abc1.slice/crio-58af22cadd22fb494a38489dcb522e03dab3494597fffe1b9913257375cd3c51 WatchSource:0}: Error finding container 58af22cadd22fb494a38489dcb522e03dab3494597fffe1b9913257375cd3c51: Status 404 returned error can't find the container with id 58af22cadd22fb494a38489dcb522e03dab3494597fffe1b9913257375cd3c51 Jul 06 19:32:14.153998 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:14.153980 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:32:14.277911 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:14.277831 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" event={"ID":"c20c57bc-8267-4f52-8608-f2e773a1abc1","Type":"ContainerStarted","Data":"58af22cadd22fb494a38489dcb522e03dab3494597fffe1b9913257375cd3c51"} Jul 06 19:32:15.286218 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.286181 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" event={"ID":"c20c57bc-8267-4f52-8608-f2e773a1abc1","Type":"ContainerStarted","Data":"6e0025d1256bdc74b50dcc5fc9f717b10b598ac51df730649ad9d28afb43b126"} Jul 06 19:32:15.336627 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.336517 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-788c7f4f8f-6sxcn" podStartSLOduration=1.97197081 podStartE2EDuration="2.336496263s" totalImagesPullingTime="364.525453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:15.336046401 +0000 UTC m=+750.016080198" watchObservedRunningTime="2026-07-06 19:32:15.336496263 +0000 UTC m=+750.016530061" Jul 06 19:32:15.462025 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.461985 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:32:15.462251 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.462221 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" podUID="2df08284-6bef-495d-a7b0-7d5758025902" containerName="authorino" containerID="cri-o://7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6" gracePeriod=30 Jul 06 19:32:15.734567 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.734539 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:32:15.750482 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.750457 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert\") pod \"2df08284-6bef-495d-a7b0-7d5758025902\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " Jul 06 19:32:15.750603 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.750535 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwqp7\" (UniqueName: \"kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7\") pod \"2df08284-6bef-495d-a7b0-7d5758025902\" (UID: \"2df08284-6bef-495d-a7b0-7d5758025902\") " Jul 06 19:32:15.752485 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.752453 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7" (OuterVolumeSpecName: "kube-api-access-rwqp7") pod "2df08284-6bef-495d-a7b0-7d5758025902" (UID: "2df08284-6bef-495d-a7b0-7d5758025902"). InnerVolumeSpecName "kube-api-access-rwqp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:15.759965 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.759943 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2df08284-6bef-495d-a7b0-7d5758025902" (UID: "2df08284-6bef-495d-a7b0-7d5758025902"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:32:15.851027 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.851004 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwqp7\" (UniqueName: \"kubernetes.io/projected/2df08284-6bef-495d-a7b0-7d5758025902-kube-api-access-rwqp7\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:32:15.851027 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:15.851025 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2df08284-6bef-495d-a7b0-7d5758025902-tls-cert\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:32:16.294722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.294689 2576 generic.go:356] "Generic (PLEG): container finished" podID="2df08284-6bef-495d-a7b0-7d5758025902" containerID="7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6" exitCode=0 Jul 06 19:32:16.295170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.294738 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" Jul 06 19:32:16.295170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.294767 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" event={"ID":"2df08284-6bef-495d-a7b0-7d5758025902","Type":"ContainerDied","Data":"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6"} Jul 06 19:32:16.295170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.294811 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb5cddc88-7r42j" event={"ID":"2df08284-6bef-495d-a7b0-7d5758025902","Type":"ContainerDied","Data":"419989d75ff9fb739fe40e7188e15ad98158da099f331ac34cefcef8bf663874"} Jul 06 19:32:16.295170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.294832 2576 scope.go:117] "RemoveContainer" containerID="7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6" Jul 06 19:32:16.305326 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.305305 2576 scope.go:117] "RemoveContainer" containerID="7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6" Jul 06 19:32:16.305657 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:32:16.305636 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6\": container with ID starting with 7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6 not found: ID does not exist" containerID="7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6" Jul 06 19:32:16.305751 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.305661 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6"} err="failed to get container status \"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6\": rpc error: code = NotFound desc = could not find container \"7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6\": container with ID starting with 7196fdd626805a76d67c609b7367a6d68954d496d7ea5c7f4a3a2ca9f3ad8ef6 not found: ID does not exist" Jul 06 19:32:16.320941 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.320912 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:32:16.323010 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:16.322987 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fb5cddc88-7r42j"] Jul 06 19:32:17.849089 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:17.849056 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df08284-6bef-495d-a7b0-7d5758025902" path="/var/lib/kubelet/pods/2df08284-6bef-495d-a7b0-7d5758025902/volumes" Jul 06 19:32:42.432645 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:42.432606 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:32:52.123709 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:32:52.123674 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:33:03.527213 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:33:03.527180 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:33:12.222204 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:33:12.222113 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:33:23.420333 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:33:23.420292 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:33:48.938958 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:33:48.938923 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:34:01.735266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:01.735225 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:34:34.628582 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:34.628548 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:34:41.726526 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:41.726440 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:34:45.744759 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:45.744732 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:34:45.745939 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:45.745919 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:34:51.622470 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:51.622441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:34:58.627471 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:34:58.627432 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:35:07.824932 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:35:07.824892 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:35:18.340860 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:35:18.340818 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:35:29.413281 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:35:29.413250 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:35:38.619637 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:35:38.619603 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:36:01.325545 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:36:01.325505 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:36:11.219339 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:36:11.219262 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:36:34.935572 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:36:34.935542 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:36:45.619491 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:36:45.619458 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:37:01.121161 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:37:01.121113 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:37:10.034912 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:37:10.034875 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:37:20.719829 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:37:20.719795 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:37:29.831111 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:37:29.831078 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:37:38.448881 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:37:38.448806 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:39:26.827143 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:39:26.827108 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:39:35.858644 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:39:35.858605 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:39:45.775872 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:39:45.775844 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:39:45.780659 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:39:45.780636 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:39:49.521317 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:39:49.521282 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:00.242552 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:00.242515 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:11.559404 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:11.559367 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:23.632163 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:23.632115 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:31.734524 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:31.734491 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:42.132840 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:42.132760 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:40:53.154664 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:40:53.154618 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:00.439300 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:00.439267 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:09.227605 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:09.227570 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:15.539759 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:15.539722 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:36.840540 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:36.840506 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:46.167723 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:46.167690 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:41:57.151544 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:41:57.151504 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:42:09.558199 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:42:09.558113 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:42:27.350500 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:42:27.350467 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:42:41.163291 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:42:41.163256 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:42:49.951612 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:42:49.951577 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:43:04.245379 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:43:04.245345 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:43:13.074518 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:43:13.074480 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:43:42.438815 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:43:42.438734 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:43:51.648305 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:43:51.648271 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:00.882255 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:00.882225 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:12.448968 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:12.448935 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:22.440662 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:22.440628 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:35.554186 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:35.554153 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:44.055478 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:44.055443 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:44:45.809499 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:45.809470 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:44:45.819977 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:45.819948 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:44:55.043910 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:44:55.043874 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:45:05.861199 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:45:05.861166 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:45:18.667489 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:45:18.667453 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:45:27.088044 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:45:27.088010 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:45:35.840714 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:45:35.840679 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:45:56.590010 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:45:56.589970 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:46:12.040194 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:46:12.040162 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:46:18.450066 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:46:18.450033 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:46:25.530357 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:46:25.530324 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:46:37.745244 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:46:37.745208 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:48:17.158120 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:48:17.158028 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:48:26.057114 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:48:26.057079 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:48:40.135835 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:48:40.135797 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:48:51.022697 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:48:51.022657 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:49:23.544812 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:49:23.544777 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:49:37.239110 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:49:37.239075 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:49:45.844496 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:49:45.844468 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:49:45.857784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:49:45.857763 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:50:03.663329 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:50:03.663296 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:50:17.774943 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:50:17.774905 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:50:56.630117 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:50:56.630075 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:51:11.547444 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:51:11.547360 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:51:21.056910 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:51:21.056874 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:52:09.430547 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:09.430514 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:52:19.745148 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:19.745107 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:52:26.155059 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.155025 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:52:26.155541 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.155389 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df08284-6bef-495d-a7b0-7d5758025902" containerName="authorino" Jul 06 19:52:26.155541 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.155406 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df08284-6bef-495d-a7b0-7d5758025902" containerName="authorino" Jul 06 19:52:26.155541 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.155513 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df08284-6bef-495d-a7b0-7d5758025902" containerName="authorino" Jul 06 19:52:26.158534 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.158517 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.161587 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.161563 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-905adb\"/\"default-dockercfg-sx9d7\"" Jul 06 19:52:26.161823 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.161809 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-905adb\"/\"kube-root-ca.crt\"" Jul 06 19:52:26.161894 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.161878 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-905adb\"/\"test-model-905adb-kserve-self-signed-certs\"" Jul 06 19:52:26.162283 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.162269 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-905adb\"/\"openshift-service-ca.crt\"" Jul 06 19:52:26.181051 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.181028 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:52:26.186643 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.186739 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.186788 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186743 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlp9q\" (UniqueName: \"kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.186827 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.186860 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.186896 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.186871 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.287783 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlp9q\" (UniqueName: \"kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.287953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.287953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.287953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.287953 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287919 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.288209 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.287968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.288209 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.288188 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.288313 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.288257 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.288353 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.288320 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.290043 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.290016 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.290421 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.290401 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.299244 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.299223 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlp9q\" (UniqueName: \"kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q\") pod \"test-model-905adb-kserve-646cc57b67-pnj5k\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.467990 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.467954 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:26.607341 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.607307 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:52:26.610181 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:52:26.610155 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89aaa960_67c1_4a98_b5a6_c637d8b03a31.slice/crio-048855b1191c65f333651a970c7112298a7f3b3dcbbc1b33227cb463ed146b82 WatchSource:0}: Error finding container 048855b1191c65f333651a970c7112298a7f3b3dcbbc1b33227cb463ed146b82: Status 404 returned error can't find the container with id 048855b1191c65f333651a970c7112298a7f3b3dcbbc1b33227cb463ed146b82 Jul 06 19:52:26.612013 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:26.611997 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:52:27.222546 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:27.222514 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" event={"ID":"89aaa960-67c1-4a98-b5a6-c637d8b03a31","Type":"ContainerStarted","Data":"048855b1191c65f333651a970c7112298a7f3b3dcbbc1b33227cb463ed146b82"} Jul 06 19:52:28.231155 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:28.231101 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" event={"ID":"89aaa960-67c1-4a98-b5a6-c637d8b03a31","Type":"ContainerStarted","Data":"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441"} Jul 06 19:52:28.256488 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:28.256444 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" podStartSLOduration=1.13450987 podStartE2EDuration="2.25643001s" totalImagesPullingTime="1.12192014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:52:28.25630464 +0000 UTC m=+1962.936338436" watchObservedRunningTime="2026-07-06 19:52:28.25643001 +0000 UTC m=+1962.936463784" Jul 06 19:52:36.468486 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:36.468447 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:36.468964 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:36.468508 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:36.481225 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:36.481202 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:37.287446 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:37.287416 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:52:37.426245 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:37.426209 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:52:55.140739 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:52:55.140700 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:53:16.531247 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.531216 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:16.536729 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.536710 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.539400 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.539379 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-reuse-bdf3a5-serving-cert\"" Jul 06 19:53:16.539513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.539437 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lvxq\"" Jul 06 19:53:16.548210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.548189 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:16.613043 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.613016 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.613043 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.613051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.613254 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.613075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9bbw\" (UniqueName: \"kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.714023 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.713992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.714177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.714027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.714177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.714051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9bbw\" (UniqueName: \"kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.714177 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:53:16.714121 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-reuse-bdf3a5-serving-cert: secret "maas-api-e2e-reuse-bdf3a5-serving-cert" not found Jul 06 19:53:16.714283 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:53:16.714208 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls podName:76781d0b-3d98-4990-a24d-aeae46494992 nodeName:}" failed. No retries permitted until 2026-07-06 19:53:17.214191518 +0000 UTC m=+2011.894225297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls") pod "maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" (UID: "76781d0b-3d98-4990-a24d-aeae46494992") : secret "maas-api-e2e-reuse-bdf3a5-serving-cert" not found Jul 06 19:53:16.714639 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.714622 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:16.736765 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:16.736735 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9bbw\" (UniqueName: \"kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:17.218528 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:17.218489 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:17.220965 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:17.220939 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") pod \"maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:17.446460 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:17.446430 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:17.595041 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:17.595017 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:17.597488 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:53:17.597460 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76781d0b_3d98_4990_a24d_aeae46494992.slice/crio-a815bc4764e5e85a8661a39d842bd9b2dc102ee5cbb428536ecb2718a82583ac WatchSource:0}: Error finding container a815bc4764e5e85a8661a39d842bd9b2dc102ee5cbb428536ecb2718a82583ac: Status 404 returned error can't find the container with id a815bc4764e5e85a8661a39d842bd9b2dc102ee5cbb428536ecb2718a82583ac Jul 06 19:53:18.481507 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:18.481464 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" event={"ID":"76781d0b-3d98-4990-a24d-aeae46494992","Type":"ContainerStarted","Data":"a815bc4764e5e85a8661a39d842bd9b2dc102ee5cbb428536ecb2718a82583ac"} Jul 06 19:53:19.488210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:19.488167 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" event={"ID":"76781d0b-3d98-4990-a24d-aeae46494992","Type":"ContainerStarted","Data":"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3"} Jul 06 19:53:19.488549 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:19.488246 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:19.505649 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:19.505603 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" podStartSLOduration=1.7960281679999999 podStartE2EDuration="3.505588733s" totalImagesPullingTime="1.709560565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:19.504802022 +0000 UTC m=+2014.184835819" watchObservedRunningTime="2026-07-06 19:53:19.505588733 +0000 UTC m=+2014.185622530" Jul 06 19:53:25.500152 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:25.500106 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:30.616804 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:30.616770 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:53:46.529080 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:46.529044 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:53:47.928173 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:47.928124 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:47.929295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:47.929109 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" podUID="76781d0b-3d98-4990-a24d-aeae46494992" containerName="maas-api" containerID="cri-o://464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3" gracePeriod=30 Jul 06 19:53:48.170617 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.170593 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:48.262093 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.262066 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") pod \"76781d0b-3d98-4990-a24d-aeae46494992\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " Jul 06 19:53:48.262250 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.262155 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9bbw\" (UniqueName: \"kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw\") pod \"76781d0b-3d98-4990-a24d-aeae46494992\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " Jul 06 19:53:48.262250 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.262207 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca\") pod \"76781d0b-3d98-4990-a24d-aeae46494992\" (UID: \"76781d0b-3d98-4990-a24d-aeae46494992\") " Jul 06 19:53:48.262550 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.262520 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76781d0b-3d98-4990-a24d-aeae46494992" (UID: "76781d0b-3d98-4990-a24d-aeae46494992"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:53:48.264226 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.264205 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76781d0b-3d98-4990-a24d-aeae46494992" (UID: "76781d0b-3d98-4990-a24d-aeae46494992"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:53:48.264335 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.264314 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw" (OuterVolumeSpecName: "kube-api-access-v9bbw") pod "76781d0b-3d98-4990-a24d-aeae46494992" (UID: "76781d0b-3d98-4990-a24d-aeae46494992"). InnerVolumeSpecName "kube-api-access-v9bbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:53:48.362847 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.362818 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9bbw\" (UniqueName: \"kubernetes.io/projected/76781d0b-3d98-4990-a24d-aeae46494992-kube-api-access-v9bbw\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:53:48.362847 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.362844 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76781d0b-3d98-4990-a24d-aeae46494992-openshift-service-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:53:48.363009 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.362854 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76781d0b-3d98-4990-a24d-aeae46494992-maas-api-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:53:48.637669 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.637579 2576 generic.go:356] "Generic (PLEG): container finished" podID="76781d0b-3d98-4990-a24d-aeae46494992" containerID="464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3" exitCode=0 Jul 06 19:53:48.637669 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.637625 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" Jul 06 19:53:48.637855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.637662 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" event={"ID":"76781d0b-3d98-4990-a24d-aeae46494992","Type":"ContainerDied","Data":"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3"} Jul 06 19:53:48.637855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.637719 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p" event={"ID":"76781d0b-3d98-4990-a24d-aeae46494992","Type":"ContainerDied","Data":"a815bc4764e5e85a8661a39d842bd9b2dc102ee5cbb428536ecb2718a82583ac"} Jul 06 19:53:48.637855 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.637741 2576 scope.go:117] "RemoveContainer" containerID="464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3" Jul 06 19:53:48.647324 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.647301 2576 scope.go:117] "RemoveContainer" containerID="464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3" Jul 06 19:53:48.647604 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:53:48.647580 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3\": container with ID starting with 464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3 not found: ID does not exist" containerID="464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3" Jul 06 19:53:48.647673 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.647615 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3"} err="failed to get container status \"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3\": rpc error: code = NotFound desc = could not find container \"464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3\": container with ID starting with 464e36a33b87820310de80d2bbc9e07915691abc1f5303470d068c479227e5c3 not found: ID does not exist" Jul 06 19:53:48.662295 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.662272 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:48.669387 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:48.669367 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-reuse-bdf3a5-5bf879f7f9-2lx5p"] Jul 06 19:53:49.849941 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:53:49.849908 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76781d0b-3d98-4990-a24d-aeae46494992" path="/var/lib/kubelet/pods/76781d0b-3d98-4990-a24d-aeae46494992/volumes" Jul 06 19:54:04.632900 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:04.632866 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:54:34.039446 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:34.039416 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:54:37.900595 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:37.900559 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:54:45.879560 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:45.879529 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:54:45.889946 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:45.889923 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:54:49.636233 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:49.636196 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:54:52.920470 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:52.920432 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" podUID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" containerName="main" containerID="cri-o://f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441" gracePeriod=45 Jul 06 19:54:53.167878 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.167848 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:54:53.301721 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301686 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlp9q\" (UniqueName: \"kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.301721 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301725 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.301960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301771 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.301960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301833 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.301960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301863 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.301960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.301906 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache\") pod \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\" (UID: \"89aaa960-67c1-4a98-b5a6-c637d8b03a31\") " Jul 06 19:54:53.302200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.302105 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home" (OuterVolumeSpecName: "home") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.302200 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.302178 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.302280 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.302206 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache" (OuterVolumeSpecName: "model-cache") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.304060 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.304035 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm" (OuterVolumeSpecName: "dshm") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:54:53.304170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.304065 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:54:53.304170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.304144 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q" (OuterVolumeSpecName: "kube-api-access-mlp9q") pod "89aaa960-67c1-4a98-b5a6-c637d8b03a31" (UID: "89aaa960-67c1-4a98-b5a6-c637d8b03a31"). InnerVolumeSpecName "kube-api-access-mlp9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:54:53.403178 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403125 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tls-certs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.403178 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403175 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-dshm\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.403178 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403183 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-home\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.403395 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403190 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-tmp-dir\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.403395 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403198 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aaa960-67c1-4a98-b5a6-c637d8b03a31-model-cache\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.403395 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.403207 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlp9q\" (UniqueName: \"kubernetes.io/projected/89aaa960-67c1-4a98-b5a6-c637d8b03a31-kube-api-access-mlp9q\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:54:53.957074 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.957045 2576 generic.go:356] "Generic (PLEG): container finished" podID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" containerID="f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441" exitCode=0 Jul 06 19:54:53.957512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.957086 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" Jul 06 19:54:53.957512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.957112 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" event={"ID":"89aaa960-67c1-4a98-b5a6-c637d8b03a31","Type":"ContainerDied","Data":"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441"} Jul 06 19:54:53.957512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.957170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k" event={"ID":"89aaa960-67c1-4a98-b5a6-c637d8b03a31","Type":"ContainerDied","Data":"048855b1191c65f333651a970c7112298a7f3b3dcbbc1b33227cb463ed146b82"} Jul 06 19:54:53.957512 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.957187 2576 scope.go:117] "RemoveContainer" containerID="f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441" Jul 06 19:54:53.968602 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.968534 2576 scope.go:117] "RemoveContainer" containerID="f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441" Jul 06 19:54:53.969064 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:54:53.969034 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441\": container with ID starting with f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441 not found: ID does not exist" containerID="f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441" Jul 06 19:54:53.969243 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.969063 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441"} err="failed to get container status \"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441\": rpc error: code = NotFound desc = could not find container \"f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441\": container with ID starting with f2f3de02aa6b1bc6f7b7ece1daee169769c66df91324e761465a827bd3a35441 not found: ID does not exist" Jul 06 19:54:53.975624 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.975601 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:54:53.979624 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:53.979602 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-905adb/test-model-905adb-kserve-646cc57b67-pnj5k"] Jul 06 19:54:55.850165 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:54:55.850116 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" path="/var/lib/kubelet/pods/89aaa960-67c1-4a98-b5a6-c637d8b03a31/volumes" Jul 06 19:55:25.621839 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:25.621806 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:55:33.899906 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.899870 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:33.900285 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900207 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76781d0b-3d98-4990-a24d-aeae46494992" containerName="maas-api" Jul 06 19:55:33.900285 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900218 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="76781d0b-3d98-4990-a24d-aeae46494992" containerName="maas-api" Jul 06 19:55:33.900285 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900232 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" containerName="main" Jul 06 19:55:33.900285 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900239 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" containerName="main" Jul 06 19:55:33.900413 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900289 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89aaa960-67c1-4a98-b5a6-c637d8b03a31" containerName="main" Jul 06 19:55:33.900413 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.900300 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="76781d0b-3d98-4990-a24d-aeae46494992" containerName="maas-api" Jul 06 19:55:33.903494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.903478 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:33.906880 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.906856 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lvxq\"" Jul 06 19:55:33.907110 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.907093 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-a5ae2d-serving-cert\"" Jul 06 19:55:33.919220 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:33.919197 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:34.028439 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.028402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsm2\" (UniqueName: \"kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.028621 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.028442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.028621 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.028479 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.129322 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.129288 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsm2\" (UniqueName: \"kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.129322 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.129325 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.129513 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.129345 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.129513 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:34.129459 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-a5ae2d-serving-cert: secret "maas-api-e2e-api-a-a5ae2d-serving-cert" not found Jul 06 19:55:34.129616 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:34.129522 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls podName:e94a269f-1539-4261-b2db-4bc4e52ed19f nodeName:}" failed. No retries permitted until 2026-07-06 19:55:34.629505668 +0000 UTC m=+2149.309539442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls") pod "maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" (UID: "e94a269f-1539-4261-b2db-4bc4e52ed19f") : secret "maas-api-e2e-api-a-a5ae2d-serving-cert" not found Jul 06 19:55:34.129954 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.129931 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.151109 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.151060 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsm2\" (UniqueName: \"kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.634908 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:34.634872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:34.635091 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:34.635028 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-a5ae2d-serving-cert: secret "maas-api-e2e-api-a-a5ae2d-serving-cert" not found Jul 06 19:55:34.635170 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:34.635103 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls podName:e94a269f-1539-4261-b2db-4bc4e52ed19f nodeName:}" failed. No retries permitted until 2026-07-06 19:55:35.635087242 +0000 UTC m=+2150.315121016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls") pod "maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" (UID: "e94a269f-1539-4261-b2db-4bc4e52ed19f") : secret "maas-api-e2e-api-a-a5ae2d-serving-cert" not found Jul 06 19:55:35.644464 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:35.644432 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:35.646803 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:35.646783 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:35.713729 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:35.713692 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:35.848505 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:55:35.848464 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94a269f_1539_4261_b2db_4bc4e52ed19f.slice/crio-3f6d9ebf61251ad369b6650c9e4a57f9d67a4fe8e4186f5522caa233582bff35 WatchSource:0}: Error finding container 3f6d9ebf61251ad369b6650c9e4a57f9d67a4fe8e4186f5522caa233582bff35: Status 404 returned error can't find the container with id 3f6d9ebf61251ad369b6650c9e4a57f9d67a4fe8e4186f5522caa233582bff35 Jul 06 19:55:35.850293 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:35.850276 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:36.172237 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:36.172198 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" event={"ID":"e94a269f-1539-4261-b2db-4bc4e52ed19f","Type":"ContainerStarted","Data":"3f6d9ebf61251ad369b6650c9e4a57f9d67a4fe8e4186f5522caa233582bff35"} Jul 06 19:55:37.184784 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:37.184747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" event={"ID":"e94a269f-1539-4261-b2db-4bc4e52ed19f","Type":"ContainerStarted","Data":"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080"} Jul 06 19:55:37.185193 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:37.184833 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:37.205324 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:37.205279 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" podStartSLOduration=3.699779923 podStartE2EDuration="4.20526334s" totalImagesPullingTime="505.483417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:37.202611581 +0000 UTC m=+2151.882645574" watchObservedRunningTime="2026-07-06 19:55:37.20526334 +0000 UTC m=+2151.885297158" Jul 06 19:55:39.149325 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:39.149292 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:55:42.267863 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.267818 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:55:42.271479 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.271460 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.273959 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.273940 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-bab069-serving-cert\"" Jul 06 19:55:42.282396 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.282376 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:55:42.294523 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.294492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z246l\" (UniqueName: \"kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.294612 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.294598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.294669 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.294653 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.395666 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.395625 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.395879 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.395708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z246l\" (UniqueName: \"kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.395879 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.395763 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.395879 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:42.395791 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-bab069-serving-cert: secret "maas-api-e2e-api-b-bab069-serving-cert" not found Jul 06 19:55:42.395879 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:42.395862 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls podName:6ee76a07-f8c7-4592-b675-ffb29e08156f nodeName:}" failed. No retries permitted until 2026-07-06 19:55:42.895846068 +0000 UTC m=+2157.575879842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls") pod "maas-api-e2e-api-b-bab069-658595df4c-cc9v5" (UID: "6ee76a07-f8c7-4592-b675-ffb29e08156f") : secret "maas-api-e2e-api-b-bab069-serving-cert" not found Jul 06 19:55:42.396409 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.396392 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.409371 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.409341 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z246l\" (UniqueName: \"kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.899924 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.899888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:42.902327 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:42.902296 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") pod \"maas-api-e2e-api-b-bab069-658595df4c-cc9v5\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:43.184068 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:43.184038 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:43.195087 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:43.195060 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:43.328115 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:43.328091 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:55:43.329375 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:55:43.329350 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee76a07_f8c7_4592_b675_ffb29e08156f.slice/crio-5b9c9b113e34efe2d2e9eef38a225f8ddba6120bcd60ad1708ef08c8b031fca1 WatchSource:0}: Error finding container 5b9c9b113e34efe2d2e9eef38a225f8ddba6120bcd60ad1708ef08c8b031fca1: Status 404 returned error can't find the container with id 5b9c9b113e34efe2d2e9eef38a225f8ddba6120bcd60ad1708ef08c8b031fca1 Jul 06 19:55:44.222827 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:44.222795 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" event={"ID":"6ee76a07-f8c7-4592-b675-ffb29e08156f","Type":"ContainerStarted","Data":"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4"} Jul 06 19:55:44.223003 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:44.222846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" event={"ID":"6ee76a07-f8c7-4592-b675-ffb29e08156f","Type":"ContainerStarted","Data":"5b9c9b113e34efe2d2e9eef38a225f8ddba6120bcd60ad1708ef08c8b031fca1"} Jul 06 19:55:44.223003 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:44.222938 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:44.242232 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:44.242188 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" podStartSLOduration=1.937298824 podStartE2EDuration="2.242171689s" totalImagesPullingTime="304.872865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:44.240499823 +0000 UTC m=+2158.920533621" watchObservedRunningTime="2026-07-06 19:55:44.242171689 +0000 UTC m=+2158.922205486" Jul 06 19:55:49.528096 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:49.524695 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:55:50.233463 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:50.233437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:55:56.029757 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.029720 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:56.030369 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.029975 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" podUID="e94a269f-1539-4261-b2db-4bc4e52ed19f" containerName="maas-api" containerID="cri-o://911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080" gracePeriod=30 Jul 06 19:55:56.277282 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.277254 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:56.282933 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.282878 2576 generic.go:356] "Generic (PLEG): container finished" podID="e94a269f-1539-4261-b2db-4bc4e52ed19f" containerID="911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080" exitCode=0 Jul 06 19:55:56.282933 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.282913 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" Jul 06 19:55:56.283109 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.282963 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" event={"ID":"e94a269f-1539-4261-b2db-4bc4e52ed19f","Type":"ContainerDied","Data":"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080"} Jul 06 19:55:56.283109 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.283029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff" event={"ID":"e94a269f-1539-4261-b2db-4bc4e52ed19f","Type":"ContainerDied","Data":"3f6d9ebf61251ad369b6650c9e4a57f9d67a4fe8e4186f5522caa233582bff35"} Jul 06 19:55:56.283109 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.283058 2576 scope.go:117] "RemoveContainer" containerID="911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080" Jul 06 19:55:56.291244 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.291230 2576 scope.go:117] "RemoveContainer" containerID="911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080" Jul 06 19:55:56.291453 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:55:56.291437 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080\": container with ID starting with 911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080 not found: ID does not exist" containerID="911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080" Jul 06 19:55:56.291502 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.291461 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080"} err="failed to get container status \"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080\": rpc error: code = NotFound desc = could not find container \"911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080\": container with ID starting with 911ab0f12121044679dd0a3654ab37a898b17928d77d6c97410ad3eaf0d0d080 not found: ID does not exist" Jul 06 19:55:56.319457 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.319434 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca\") pod \"e94a269f-1539-4261-b2db-4bc4e52ed19f\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " Jul 06 19:55:56.319556 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.319475 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") pod \"e94a269f-1539-4261-b2db-4bc4e52ed19f\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " Jul 06 19:55:56.319556 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.319506 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgsm2\" (UniqueName: \"kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2\") pod \"e94a269f-1539-4261-b2db-4bc4e52ed19f\" (UID: \"e94a269f-1539-4261-b2db-4bc4e52ed19f\") " Jul 06 19:55:56.319861 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.319832 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e94a269f-1539-4261-b2db-4bc4e52ed19f" (UID: "e94a269f-1539-4261-b2db-4bc4e52ed19f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:55:56.321522 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.321496 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e94a269f-1539-4261-b2db-4bc4e52ed19f" (UID: "e94a269f-1539-4261-b2db-4bc4e52ed19f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:55:56.321609 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.321517 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2" (OuterVolumeSpecName: "kube-api-access-rgsm2") pod "e94a269f-1539-4261-b2db-4bc4e52ed19f" (UID: "e94a269f-1539-4261-b2db-4bc4e52ed19f"). InnerVolumeSpecName "kube-api-access-rgsm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:56.419991 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.419962 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94a269f-1539-4261-b2db-4bc4e52ed19f-maas-api-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:55:56.419991 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.419987 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgsm2\" (UniqueName: \"kubernetes.io/projected/e94a269f-1539-4261-b2db-4bc4e52ed19f-kube-api-access-rgsm2\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:55:56.420198 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.419999 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94a269f-1539-4261-b2db-4bc4e52ed19f-openshift-service-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:55:56.603765 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.603732 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:56.607934 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:56.607908 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5ae2d-84f74bb5f4-88sff"] Jul 06 19:55:57.850087 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:55:57.850054 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e94a269f-1539-4261-b2db-4bc4e52ed19f" path="/var/lib/kubelet/pods/e94a269f-1539-4261-b2db-4bc4e52ed19f/volumes" Jul 06 19:56:03.424705 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.424668 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:56:03.650439 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.650409 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:56:03.650658 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.650633 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" podUID="6ee76a07-f8c7-4592-b675-ffb29e08156f" containerName="maas-api" containerID="cri-o://0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4" gracePeriod=30 Jul 06 19:56:03.895106 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.895082 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:56:03.981403 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.981378 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") pod \"6ee76a07-f8c7-4592-b675-ffb29e08156f\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " Jul 06 19:56:03.981553 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.981413 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca\") pod \"6ee76a07-f8c7-4592-b675-ffb29e08156f\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " Jul 06 19:56:03.981553 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.981500 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z246l\" (UniqueName: \"kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l\") pod \"6ee76a07-f8c7-4592-b675-ffb29e08156f\" (UID: \"6ee76a07-f8c7-4592-b675-ffb29e08156f\") " Jul 06 19:56:03.981834 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.981810 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ee76a07-f8c7-4592-b675-ffb29e08156f" (UID: "6ee76a07-f8c7-4592-b675-ffb29e08156f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:56:03.983320 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.983297 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ee76a07-f8c7-4592-b675-ffb29e08156f" (UID: "6ee76a07-f8c7-4592-b675-ffb29e08156f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:56:03.983422 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:03.983397 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l" (OuterVolumeSpecName: "kube-api-access-z246l") pod "6ee76a07-f8c7-4592-b675-ffb29e08156f" (UID: "6ee76a07-f8c7-4592-b675-ffb29e08156f"). InnerVolumeSpecName "kube-api-access-z246l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:56:04.082185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.082076 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z246l\" (UniqueName: \"kubernetes.io/projected/6ee76a07-f8c7-4592-b675-ffb29e08156f-kube-api-access-z246l\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:56:04.082185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.082107 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee76a07-f8c7-4592-b675-ffb29e08156f-maas-api-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:56:04.082185 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.082116 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee76a07-f8c7-4592-b675-ffb29e08156f-openshift-service-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:56:04.324690 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.324655 2576 generic.go:356] "Generic (PLEG): container finished" podID="6ee76a07-f8c7-4592-b675-ffb29e08156f" containerID="0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4" exitCode=0 Jul 06 19:56:04.324877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.324701 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" Jul 06 19:56:04.324877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.324743 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" event={"ID":"6ee76a07-f8c7-4592-b675-ffb29e08156f","Type":"ContainerDied","Data":"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4"} Jul 06 19:56:04.324877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.324801 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5" event={"ID":"6ee76a07-f8c7-4592-b675-ffb29e08156f","Type":"ContainerDied","Data":"5b9c9b113e34efe2d2e9eef38a225f8ddba6120bcd60ad1708ef08c8b031fca1"} Jul 06 19:56:04.324877 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.324824 2576 scope.go:117] "RemoveContainer" containerID="0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4" Jul 06 19:56:04.334744 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.334727 2576 scope.go:117] "RemoveContainer" containerID="0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4" Jul 06 19:56:04.335012 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:56:04.334994 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4\": container with ID starting with 0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4 not found: ID does not exist" containerID="0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4" Jul 06 19:56:04.335053 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.335020 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4"} err="failed to get container status \"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4\": rpc error: code = NotFound desc = could not find container \"0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4\": container with ID starting with 0cfde1b3481810555112f65233e5ff250a09260fd66decb0630c59e5ade487c4 not found: ID does not exist" Jul 06 19:56:04.364092 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.364063 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:56:04.376207 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:04.376186 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-bab069-658595df4c-cc9v5"] Jul 06 19:56:05.849648 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:05.849616 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee76a07-f8c7-4592-b675-ffb29e08156f" path="/var/lib/kubelet/pods/6ee76a07-f8c7-4592-b675-ffb29e08156f/volumes" Jul 06 19:56:12.836050 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:12.836015 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:56:14.004889 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.004858 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 19:56:14.005263 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005213 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94a269f-1539-4261-b2db-4bc4e52ed19f" containerName="maas-api" Jul 06 19:56:14.005263 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005229 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94a269f-1539-4261-b2db-4bc4e52ed19f" containerName="maas-api" Jul 06 19:56:14.005263 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005250 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee76a07-f8c7-4592-b675-ffb29e08156f" containerName="maas-api" Jul 06 19:56:14.005263 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005255 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee76a07-f8c7-4592-b675-ffb29e08156f" containerName="maas-api" Jul 06 19:56:14.005396 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005312 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee76a07-f8c7-4592-b675-ffb29e08156f" containerName="maas-api" Jul 06 19:56:14.005396 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.005322 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94a269f-1539-4261-b2db-4bc4e52ed19f" containerName="maas-api" Jul 06 19:56:14.009519 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.009499 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.012104 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.012077 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-9cb33a-serving-cert\"" Jul 06 19:56:14.012249 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.012143 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lvxq\"" Jul 06 19:56:14.020581 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.020558 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 19:56:14.172546 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.172511 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnln9\" (UniqueName: \"kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.172712 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.172558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.172712 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.172584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.273492 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.273415 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnln9\" (UniqueName: \"kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.273492 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.273464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.273665 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.273502 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.273665 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:56:14.273639 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-9cb33a-serving-cert: secret "maas-api-e2e-shared-a-9cb33a-serving-cert" not found Jul 06 19:56:14.273736 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:56:14.273701 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls podName:97822741-fdd0-4be3-be40-4319db66c93c nodeName:}" failed. No retries permitted until 2026-07-06 19:56:14.773683887 +0000 UTC m=+2189.453717661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls") pod "maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" (UID: "97822741-fdd0-4be3-be40-4319db66c93c") : secret "maas-api-e2e-shared-a-9cb33a-serving-cert" not found Jul 06 19:56:14.274210 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.274194 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.282672 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.282653 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnln9\" (UniqueName: \"kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.777066 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.777026 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.779426 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.779408 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") pod \"maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:14.920240 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:14.920198 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:15.055961 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:15.055935 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 19:56:15.056667 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:56:15.056643 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97822741_fdd0_4be3_be40_4319db66c93c.slice/crio-6b52d3a542a237af0f9935275a8325a07fc07144271433f9abdb6a048e30a9cb WatchSource:0}: Error finding container 6b52d3a542a237af0f9935275a8325a07fc07144271433f9abdb6a048e30a9cb: Status 404 returned error can't find the container with id 6b52d3a542a237af0f9935275a8325a07fc07144271433f9abdb6a048e30a9cb Jul 06 19:56:15.382746 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:15.382715 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" event={"ID":"97822741-fdd0-4be3-be40-4319db66c93c","Type":"ContainerStarted","Data":"6b52d3a542a237af0f9935275a8325a07fc07144271433f9abdb6a048e30a9cb"} Jul 06 19:56:16.389817 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:16.389775 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" event={"ID":"97822741-fdd0-4be3-be40-4319db66c93c","Type":"ContainerStarted","Data":"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924"} Jul 06 19:56:16.390286 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:16.389841 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:16.412498 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:16.412453 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" podStartSLOduration=3.121335241 podStartE2EDuration="3.412438816s" totalImagesPullingTime="291.103575ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:16.410302726 +0000 UTC m=+2191.090336536" watchObservedRunningTime="2026-07-06 19:56:16.412438816 +0000 UTC m=+2191.092472612" Jul 06 19:56:22.400304 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.400276 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 19:56:22.518218 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.518181 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 19:56:22.521484 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.521461 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.523882 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.523866 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-97cecc-serving-cert\"" Jul 06 19:56:22.533390 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.533364 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 19:56:22.643307 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.643265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.643492 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.643320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th45m\" (UniqueName: \"kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.643492 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.643398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.744503 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.744472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.744704 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.744517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th45m\" (UniqueName: \"kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.744704 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.744549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.744704 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:56:22.744650 2576 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-97cecc-serving-cert: secret "maas-api-e2e-shared-b-97cecc-serving-cert" not found Jul 06 19:56:22.744875 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:56:22.744727 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls podName:4886fe04-08d5-4ceb-9963-0080a670be42 nodeName:}" failed. No retries permitted until 2026-07-06 19:56:23.244706352 +0000 UTC m=+2197.924740126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls") pod "maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" (UID: "4886fe04-08d5-4ceb-9963-0080a670be42") : secret "maas-api-e2e-shared-b-97cecc-serving-cert" not found Jul 06 19:56:22.745195 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.745179 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:22.754925 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:22.754904 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th45m\" (UniqueName: \"kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:23.249016 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:23.248982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:23.251367 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:23.251344 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") pod \"maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:23.334716 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:23.334679 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:56:23.432003 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:23.431972 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:23.570538 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:23.570514 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 19:56:23.572201 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:56:23.572177 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4886fe04_08d5_4ceb_9963_0080a670be42.slice/crio-5f192277ca879b5ee2eda0b30c24b93c32e1aaac6bcc4401a8ddb654913e7cae WatchSource:0}: Error finding container 5f192277ca879b5ee2eda0b30c24b93c32e1aaac6bcc4401a8ddb654913e7cae: Status 404 returned error can't find the container with id 5f192277ca879b5ee2eda0b30c24b93c32e1aaac6bcc4401a8ddb654913e7cae Jul 06 19:56:24.430327 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:24.430300 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" event={"ID":"4886fe04-08d5-4ceb-9963-0080a670be42","Type":"ContainerStarted","Data":"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f"} Jul 06 19:56:24.430465 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:24.430352 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" event={"ID":"4886fe04-08d5-4ceb-9963-0080a670be42","Type":"ContainerStarted","Data":"5f192277ca879b5ee2eda0b30c24b93c32e1aaac6bcc4401a8ddb654913e7cae"} Jul 06 19:56:24.430465 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:24.430372 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:24.450671 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:24.450617 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" podStartSLOduration=1.906289592 podStartE2EDuration="2.45059954s" totalImagesPullingTime="544.309948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:24.449833216 +0000 UTC m=+2199.129867014" watchObservedRunningTime="2026-07-06 19:56:24.45059954 +0000 UTC m=+2199.130633336" Jul 06 19:56:30.440479 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:30.440449 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 19:56:33.421768 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:33.421733 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:56:45.623584 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:45.623552 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:56:55.517029 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.516998 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:56:55.520337 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.520321 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.523014 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.522993 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"openshift-service-ca.crt\"" Jul 06 19:56:55.524201 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.524179 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"auth-test-model-97cecc-kserve-self-signed-certs\"" Jul 06 19:56:55.524317 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.524198 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"default-dockercfg-9hsqn\"" Jul 06 19:56:55.524317 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.524213 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"kube-root-ca.crt\"" Jul 06 19:56:55.532775 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.532744 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:56:55.606901 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.606868 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.606901 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.606903 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.607170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.606929 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.607170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.607022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkm47\" (UniqueName: \"kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.607170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.607072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.607170 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.607098 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708558 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708571 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkm47\" (UniqueName: \"kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708598 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708622 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.708755 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.708715 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.709053 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.709024 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.709100 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.709049 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.709100 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.709074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.710902 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.710873 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.711177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.711157 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.717154 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.717105 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkm47\" (UniqueName: \"kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47\") pod \"auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.829622 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.829555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:56:55.963410 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:55.963382 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:56:55.965385 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:56:55.965355 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1378ed65_e798_4e61_9f5d_60ad5b1abc98.slice/crio-65ed6e6f8d0779de135557dddb280f1d127a11bfe19ea4452d1658b6d4168156 WatchSource:0}: Error finding container 65ed6e6f8d0779de135557dddb280f1d127a11bfe19ea4452d1658b6d4168156: Status 404 returned error can't find the container with id 65ed6e6f8d0779de135557dddb280f1d127a11bfe19ea4452d1658b6d4168156 Jul 06 19:56:56.587733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:56.587693 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" event={"ID":"1378ed65-e798-4e61-9f5d-60ad5b1abc98","Type":"ContainerStarted","Data":"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3"} Jul 06 19:56:56.587733 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:56.587740 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" event={"ID":"1378ed65-e798-4e61-9f5d-60ad5b1abc98","Type":"ContainerStarted","Data":"65ed6e6f8d0779de135557dddb280f1d127a11bfe19ea4452d1658b6d4168156"} Jul 06 19:56:56.607978 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:56.607929 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" podStartSLOduration=1.607915061 podStartE2EDuration="1.607915061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:56.60705206 +0000 UTC m=+2231.287085858" watchObservedRunningTime="2026-07-06 19:56:56.607915061 +0000 UTC m=+2231.287948859" Jul 06 19:56:59.434525 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:56:59.434487 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:57:05.830597 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:05.830540 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:57:05.831105 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:05.830619 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:57:05.849960 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:05.849938 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:57:06.649416 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:06.649388 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:57:13.394102 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:13.394023 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:57:31.589398 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:31.589362 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:57:44.540266 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:57:44.540223 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:58:01.231208 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:58:01.231173 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:58:19.026863 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:58:19.026827 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:58:35.326441 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:58:35.326410 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:58:52.828874 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:58:52.828796 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:58:57.741298 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:58:57.741264 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:59:08.338494 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:08.338461 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:59:12.761083 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:12.761021 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" podUID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" containerName="main" containerID="cri-o://e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3" gracePeriod=45 Jul 06 19:59:13.003869 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.003847 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:59:13.136412 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136320 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136412 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136399 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkm47\" (UniqueName: \"kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136431 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136460 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136498 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136557 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir\") pod \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\" (UID: \"1378ed65-e798-4e61-9f5d-60ad5b1abc98\") " Jul 06 19:59:13.136630 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136588 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache" (OuterVolumeSpecName: "model-cache") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:13.136875 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136763 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home" (OuterVolumeSpecName: "home") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:13.136875 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136832 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-model-cache\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.136875 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.136858 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:13.138822 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.138798 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm" (OuterVolumeSpecName: "dshm") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:59:13.138905 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.138800 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:59:13.138905 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.138832 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47" (OuterVolumeSpecName: "kube-api-access-hkm47") pod "1378ed65-e798-4e61-9f5d-60ad5b1abc98" (UID: "1378ed65-e798-4e61-9f5d-60ad5b1abc98"). InnerVolumeSpecName "kube-api-access-hkm47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:59:13.238046 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.238020 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tmp-dir\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.238046 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.238043 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkm47\" (UniqueName: \"kubernetes.io/projected/1378ed65-e798-4e61-9f5d-60ad5b1abc98-kube-api-access-hkm47\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.238046 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.238051 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-dshm\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.238261 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.238060 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1378ed65-e798-4e61-9f5d-60ad5b1abc98-home\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.238261 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.238070 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1378ed65-e798-4e61-9f5d-60ad5b1abc98-tls-certs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 19:59:13.268475 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.268447 2576 generic.go:356] "Generic (PLEG): container finished" podID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" containerID="e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3" exitCode=0 Jul 06 19:59:13.268586 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.268488 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" Jul 06 19:59:13.268586 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.268530 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" event={"ID":"1378ed65-e798-4e61-9f5d-60ad5b1abc98","Type":"ContainerDied","Data":"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3"} Jul 06 19:59:13.268586 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.268568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp" event={"ID":"1378ed65-e798-4e61-9f5d-60ad5b1abc98","Type":"ContainerDied","Data":"65ed6e6f8d0779de135557dddb280f1d127a11bfe19ea4452d1658b6d4168156"} Jul 06 19:59:13.268586 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.268585 2576 scope.go:117] "RemoveContainer" containerID="e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3" Jul 06 19:59:13.278177 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.278160 2576 scope.go:117] "RemoveContainer" containerID="e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3" Jul 06 19:59:13.278415 ip-10-0-130-239 kubenswrapper[2576]: E0706 19:59:13.278397 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3\": container with ID starting with e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3 not found: ID does not exist" containerID="e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3" Jul 06 19:59:13.278450 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.278424 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3"} err="failed to get container status \"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3\": rpc error: code = NotFound desc = could not find container \"e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3\": container with ID starting with e2da47307c946f2101ad4d9b1a4c19a400408e9f6f3f062f7ad2828e3826c8c3 not found: ID does not exist" Jul 06 19:59:13.292832 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.292808 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:59:13.297049 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.297029 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/auth-test-model-97cecc-kserve-5c65b7bdc5-tsgfp"] Jul 06 19:59:13.848804 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:13.848769 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" path="/var/lib/kubelet/pods/1378ed65-e798-4e61-9f5d-60ad5b1abc98/volumes" Jul 06 19:59:19.325840 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:19.325807 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:59:44.127100 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.127067 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 19:59:44.127561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.127416 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" containerName="main" Jul 06 19:59:44.127561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.127429 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" containerName="main" Jul 06 19:59:44.127561 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.127496 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1378ed65-e798-4e61-9f5d-60ad5b1abc98" containerName="main" Jul 06 19:59:44.130271 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.130252 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.132790 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.132766 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"sub-test-model-97cecc-kserve-self-signed-certs\"" Jul 06 19:59:44.132920 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.132806 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"openshift-service-ca.crt\"" Jul 06 19:59:44.133764 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.133746 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"default-dockercfg-9hsqn\"" Jul 06 19:59:44.133851 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.133822 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"kube-root-ca.crt\"" Jul 06 19:59:44.140033 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.140013 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 19:59:44.197547 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197518 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.197671 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.197671 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnwws\" (UniqueName: \"kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.197671 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.197786 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.197786 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.197703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299026 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.298993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299222 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299054 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299222 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299222 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnwws\" (UniqueName: \"kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299402 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299245 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299402 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299271 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299524 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299499 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299535 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.299591 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.299571 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.301296 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.301267 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.301644 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.301625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.308966 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.308941 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnwws\" (UniqueName: \"kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws\") pod \"sub-test-model-97cecc-kserve-7df8c554dc-jmhmw\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.440498 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.440470 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:44.573697 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:44.573669 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 19:59:44.575511 ip-10-0-130-239 kubenswrapper[2576]: W0706 19:59:44.575482 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a641d04_bc6c_4ea2_849e_b49dca13af78.slice/crio-dec0790bf751c1682e53a22da21163df03c7ebb68c9306027fd34173fc3f5cee WatchSource:0}: Error finding container dec0790bf751c1682e53a22da21163df03c7ebb68c9306027fd34173fc3f5cee: Status 404 returned error can't find the container with id dec0790bf751c1682e53a22da21163df03c7ebb68c9306027fd34173fc3f5cee Jul 06 19:59:45.429867 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:45.429833 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" event={"ID":"5a641d04-bc6c-4ea2-849e-b49dca13af78","Type":"ContainerStarted","Data":"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114"} Jul 06 19:59:45.429867 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:45.429872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" event={"ID":"5a641d04-bc6c-4ea2-849e-b49dca13af78","Type":"ContainerStarted","Data":"dec0790bf751c1682e53a22da21163df03c7ebb68c9306027fd34173fc3f5cee"} Jul 06 19:59:45.451123 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:45.451080 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" podStartSLOduration=1.451065917 podStartE2EDuration="1.451065917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:59:45.448857325 +0000 UTC m=+2400.128891124" watchObservedRunningTime="2026-07-06 19:59:45.451065917 +0000 UTC m=+2400.131099714" Jul 06 19:59:45.918356 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:45.918271 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:59:45.924738 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:45.924718 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 19:59:49.050722 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:49.050678 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 19:59:54.441305 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:54.441264 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:54.441776 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:54.441315 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:54.454079 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:54.454051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 19:59:54.485526 ip-10-0-130-239 kubenswrapper[2576]: I0706 19:59:54.485506 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 20:00:08.324752 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:08.324669 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:00:22.524102 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:22.524059 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:00:28.870331 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:28.870297 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 20:00:40.125945 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:40.125907 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:00:43.889540 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:43.889496 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" podUID="5a641d04-bc6c-4ea2-849e-b49dca13af78" containerName="main" containerID="cri-o://e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114" gracePeriod=45 Jul 06 20:00:44.133808 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.133786 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 20:00:44.175117 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175047 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175117 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175117 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175304 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175176 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175304 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175207 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175304 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175254 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnwws\" (UniqueName: \"kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175304 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175285 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home\") pod \"5a641d04-bc6c-4ea2-849e-b49dca13af78\" (UID: \"5a641d04-bc6c-4ea2-849e-b49dca13af78\") " Jul 06 20:00:44.175515 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175341 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:00:44.175573 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175484 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:00:44.175573 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175547 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home" (OuterVolumeSpecName: "home") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:00:44.175671 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175593 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-tmp-dir\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.175671 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.175613 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-model-cache\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.177308 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.177287 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:00:44.177392 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.177374 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws" (OuterVolumeSpecName: "kube-api-access-cnwws") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "kube-api-access-cnwws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:44.177599 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.177582 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm" (OuterVolumeSpecName: "dshm") pod "5a641d04-bc6c-4ea2-849e-b49dca13af78" (UID: "5a641d04-bc6c-4ea2-849e-b49dca13af78"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:00:44.276766 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.276729 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a641d04-bc6c-4ea2-849e-b49dca13af78-tls-certs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.276766 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.276760 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnwws\" (UniqueName: \"kubernetes.io/projected/5a641d04-bc6c-4ea2-849e-b49dca13af78-kube-api-access-cnwws\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.276766 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.276771 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-home\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.276980 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.276779 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a641d04-bc6c-4ea2-849e-b49dca13af78-dshm\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:00:44.730843 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.730809 2576 generic.go:356] "Generic (PLEG): container finished" podID="5a641d04-bc6c-4ea2-849e-b49dca13af78" containerID="e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114" exitCode=0 Jul 06 20:00:44.731036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.730860 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" Jul 06 20:00:44.731036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.730899 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" event={"ID":"5a641d04-bc6c-4ea2-849e-b49dca13af78","Type":"ContainerDied","Data":"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114"} Jul 06 20:00:44.731036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.730944 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw" event={"ID":"5a641d04-bc6c-4ea2-849e-b49dca13af78","Type":"ContainerDied","Data":"dec0790bf751c1682e53a22da21163df03c7ebb68c9306027fd34173fc3f5cee"} Jul 06 20:00:44.731036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.730963 2576 scope.go:117] "RemoveContainer" containerID="e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114" Jul 06 20:00:44.739655 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.739642 2576 scope.go:117] "RemoveContainer" containerID="e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114" Jul 06 20:00:44.739899 ip-10-0-130-239 kubenswrapper[2576]: E0706 20:00:44.739883 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114\": container with ID starting with e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114 not found: ID does not exist" containerID="e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114" Jul 06 20:00:44.739945 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.739907 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114"} err="failed to get container status \"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114\": rpc error: code = NotFound desc = could not find container \"e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114\": container with ID starting with e3c0c8f908e64044ff2eedfa550f4d2debcaccd9b54906a166257b82e7773114 not found: ID does not exist" Jul 06 20:00:44.752503 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.752484 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 20:00:44.756445 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:44.756426 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/sub-test-model-97cecc-kserve-7df8c554dc-jmhmw"] Jul 06 20:00:45.849491 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:45.849452 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a641d04-bc6c-4ea2-849e-b49dca13af78" path="/var/lib/kubelet/pods/5a641d04-bc6c-4ea2-849e-b49dca13af78/volumes" Jul 06 20:00:58.831611 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:00:58.831580 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:01:23.049948 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.049914 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:01:23.050315 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.050302 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a641d04-bc6c-4ea2-849e-b49dca13af78" containerName="main" Jul 06 20:01:23.050315 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.050315 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a641d04-bc6c-4ea2-849e-b49dca13af78" containerName="main" Jul 06 20:01:23.050392 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.050380 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a641d04-bc6c-4ea2-849e-b49dca13af78" containerName="main" Jul 06 20:01:23.053327 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.053312 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.058333 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.058297 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"rate-test-model-97cecc-kserve-self-signed-certs\"" Jul 06 20:01:23.058462 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.058344 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"kube-root-ca.crt\"" Jul 06 20:01:23.058462 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.058358 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"openshift-service-ca.crt\"" Jul 06 20:01:23.058462 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.058368 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-97cecc\"/\"default-dockercfg-9hsqn\"" Jul 06 20:01:23.078030 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.078010 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:01:23.192885 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.192857 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.193031 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.192896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.193031 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.192975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mpw9\" (UniqueName: \"kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.193031 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.193026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.193154 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.193049 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.193154 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.193069 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294204 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294204 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294216 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294420 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mpw9\" (UniqueName: \"kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294420 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294420 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294420 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294312 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294678 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294653 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294736 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294676 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.294800 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.294742 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.296369 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.296352 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.296547 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.296529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.304119 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.304064 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mpw9\" (UniqueName: \"kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9\") pod \"rate-test-model-97cecc-kserve-b5bb849c9-6p79s\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.362036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.362012 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:23.518659 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.518620 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:01:23.521669 ip-10-0-130-239 kubenswrapper[2576]: W0706 20:01:23.521641 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98dbd9d3_6f61_4ac6_b024_fe95b7e7b8c0.slice/crio-757d3ef6f555f5c7cec4901193a65b604d2949589ca965ceb086cbee93f0a8a0 WatchSource:0}: Error finding container 757d3ef6f555f5c7cec4901193a65b604d2949589ca965ceb086cbee93f0a8a0: Status 404 returned error can't find the container with id 757d3ef6f555f5c7cec4901193a65b604d2949589ca965ceb086cbee93f0a8a0 Jul 06 20:01:23.933104 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.933066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" event={"ID":"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0","Type":"ContainerStarted","Data":"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4"} Jul 06 20:01:23.933283 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.933116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" event={"ID":"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0","Type":"ContainerStarted","Data":"757d3ef6f555f5c7cec4901193a65b604d2949589ca965ceb086cbee93f0a8a0"} Jul 06 20:01:23.955399 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:23.955164 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" podStartSLOduration=0.955119479 podStartE2EDuration="955.119479ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:23.954288356 +0000 UTC m=+2498.634322152" watchObservedRunningTime="2026-07-06 20:01:23.955119479 +0000 UTC m=+2498.635153276" Jul 06 20:01:27.032523 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:27.032488 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:01:33.362859 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:33.362813 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:33.363416 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:33.362875 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:33.375459 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:33.375436 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:34.003188 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:34.003161 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:01:40.726829 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:01:40.726734 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:02:01.232901 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:01.232865 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:02:06.232150 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:06.232104 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:02:12.434707 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:12.434673 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 20:02:12.435220 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:12.434905 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" podUID="97822741-fdd0-4be3-be40-4319db66c93c" containerName="maas-api" containerID="cri-o://c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924" gracePeriod=30 Jul 06 20:02:12.880337 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:12.880308 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 20:02:13.016272 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.016193 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") pod \"97822741-fdd0-4be3-be40-4319db66c93c\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " Jul 06 20:02:13.016272 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.016248 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca\") pod \"97822741-fdd0-4be3-be40-4319db66c93c\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " Jul 06 20:02:13.016480 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.016306 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnln9\" (UniqueName: \"kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9\") pod \"97822741-fdd0-4be3-be40-4319db66c93c\" (UID: \"97822741-fdd0-4be3-be40-4319db66c93c\") " Jul 06 20:02:13.016665 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.016641 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97822741-fdd0-4be3-be40-4319db66c93c" (UID: "97822741-fdd0-4be3-be40-4319db66c93c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:02:13.018166 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.018121 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97822741-fdd0-4be3-be40-4319db66c93c" (UID: "97822741-fdd0-4be3-be40-4319db66c93c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:02:13.018265 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.018194 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9" (OuterVolumeSpecName: "kube-api-access-nnln9") pod "97822741-fdd0-4be3-be40-4319db66c93c" (UID: "97822741-fdd0-4be3-be40-4319db66c93c"). InnerVolumeSpecName "kube-api-access-nnln9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:13.117261 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.117229 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97822741-fdd0-4be3-be40-4319db66c93c-openshift-service-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.117261 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.117253 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnln9\" (UniqueName: \"kubernetes.io/projected/97822741-fdd0-4be3-be40-4319db66c93c-kube-api-access-nnln9\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.117261 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.117265 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97822741-fdd0-4be3-be40-4319db66c93c-maas-api-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:13.189260 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.189228 2576 generic.go:356] "Generic (PLEG): container finished" podID="97822741-fdd0-4be3-be40-4319db66c93c" containerID="c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924" exitCode=0 Jul 06 20:02:13.189440 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.189264 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" Jul 06 20:02:13.189440 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.189307 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" event={"ID":"97822741-fdd0-4be3-be40-4319db66c93c","Type":"ContainerDied","Data":"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924"} Jul 06 20:02:13.189440 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.189382 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf" event={"ID":"97822741-fdd0-4be3-be40-4319db66c93c","Type":"ContainerDied","Data":"6b52d3a542a237af0f9935275a8325a07fc07144271433f9abdb6a048e30a9cb"} Jul 06 20:02:13.189440 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.189404 2576 scope.go:117] "RemoveContainer" containerID="c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924" Jul 06 20:02:13.199387 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.199364 2576 scope.go:117] "RemoveContainer" containerID="c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924" Jul 06 20:02:13.199717 ip-10-0-130-239 kubenswrapper[2576]: E0706 20:02:13.199695 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924\": container with ID starting with c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924 not found: ID does not exist" containerID="c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924" Jul 06 20:02:13.199797 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.199723 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924"} err="failed to get container status \"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924\": rpc error: code = NotFound desc = could not find container \"c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924\": container with ID starting with c7ae4dc8186ae0da395d8fae19bfe0bd21c9e10977fc57356cf1707bd73d1924 not found: ID does not exist" Jul 06 20:02:13.212320 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.212298 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 20:02:13.216236 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.216214 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-9cb33a-7d4c5b8bb6-b8fmf"] Jul 06 20:02:13.849636 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:13.849605 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97822741-fdd0-4be3-be40-4319db66c93c" path="/var/lib/kubelet/pods/97822741-fdd0-4be3-be40-4319db66c93c/volumes" Jul 06 20:02:16.219728 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:16.219694 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:02:21.251923 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.251879 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" podUID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" containerName="main" containerID="cri-o://8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4" gracePeriod=45 Jul 06 20:02:21.496666 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.496638 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:02:21.589460 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589369 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589460 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589423 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mpw9\" (UniqueName: \"kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589460 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589452 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589729 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589488 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589729 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589535 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589729 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589557 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs\") pod \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\" (UID: \"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0\") " Jul 06 20:02:21.589729 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589635 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home" (OuterVolumeSpecName: "home") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:21.589947 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589784 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:21.589947 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589815 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-home\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:21.589947 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.589819 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:21.591524 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.591502 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm" (OuterVolumeSpecName: "dshm") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:02:21.591609 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.591579 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:02:21.591680 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.591666 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9" (OuterVolumeSpecName: "kube-api-access-5mpw9") pod "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" (UID: "98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0"). InnerVolumeSpecName "kube-api-access-5mpw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:21.691071 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.691037 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tmp-dir\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:21.691071 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.691065 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-tls-certs\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:21.691264 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.691079 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mpw9\" (UniqueName: \"kubernetes.io/projected/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-kube-api-access-5mpw9\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:21.691264 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.691091 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-model-cache\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:21.691264 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:21.691102 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0-dshm\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:22.237305 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.237269 2576 generic.go:356] "Generic (PLEG): container finished" podID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" containerID="8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4" exitCode=0 Jul 06 20:02:22.237481 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.237315 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" Jul 06 20:02:22.237481 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.237354 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" event={"ID":"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0","Type":"ContainerDied","Data":"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4"} Jul 06 20:02:22.237481 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.237397 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s" event={"ID":"98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0","Type":"ContainerDied","Data":"757d3ef6f555f5c7cec4901193a65b604d2949589ca965ceb086cbee93f0a8a0"} Jul 06 20:02:22.237481 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.237416 2576 scope.go:117] "RemoveContainer" containerID="8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4" Jul 06 20:02:22.246036 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.246017 2576 scope.go:117] "RemoveContainer" containerID="8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4" Jul 06 20:02:22.246296 ip-10-0-130-239 kubenswrapper[2576]: E0706 20:02:22.246275 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4\": container with ID starting with 8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4 not found: ID does not exist" containerID="8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4" Jul 06 20:02:22.246368 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.246304 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4"} err="failed to get container status \"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4\": rpc error: code = NotFound desc = could not find container \"8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4\": container with ID starting with 8ed397ea152b6919be3dd3ed4cfea7e572c0f2bb041408f5edd7a3e7088a69f4 not found: ID does not exist" Jul 06 20:02:22.254168 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.254147 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:02:22.257876 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:22.257856 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-97cecc/rate-test-model-97cecc-kserve-b5bb849c9-6p79s"] Jul 06 20:02:23.849505 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:23.849471 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" path="/var/lib/kubelet/pods/98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0/volumes" Jul 06 20:02:24.721998 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:24.721964 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 20:02:24.722291 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:24.722267 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" podUID="4886fe04-08d5-4ceb-9963-0080a670be42" containerName="maas-api" containerID="cri-o://9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f" gracePeriod=30 Jul 06 20:02:24.961905 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:24.961881 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 20:02:25.121977 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.121886 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") pod \"4886fe04-08d5-4ceb-9963-0080a670be42\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " Jul 06 20:02:25.121977 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.121962 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca\") pod \"4886fe04-08d5-4ceb-9963-0080a670be42\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " Jul 06 20:02:25.122241 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.122007 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th45m\" (UniqueName: \"kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m\") pod \"4886fe04-08d5-4ceb-9963-0080a670be42\" (UID: \"4886fe04-08d5-4ceb-9963-0080a670be42\") " Jul 06 20:02:25.122327 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.122302 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4886fe04-08d5-4ceb-9963-0080a670be42" (UID: "4886fe04-08d5-4ceb-9963-0080a670be42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:02:25.124051 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.124027 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4886fe04-08d5-4ceb-9963-0080a670be42" (UID: "4886fe04-08d5-4ceb-9963-0080a670be42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:02:25.124109 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.124040 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m" (OuterVolumeSpecName: "kube-api-access-th45m") pod "4886fe04-08d5-4ceb-9963-0080a670be42" (UID: "4886fe04-08d5-4ceb-9963-0080a670be42"). InnerVolumeSpecName "kube-api-access-th45m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:02:25.223631 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.223599 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4886fe04-08d5-4ceb-9963-0080a670be42-maas-api-tls\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:25.223631 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.223624 2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4886fe04-08d5-4ceb-9963-0080a670be42-openshift-service-ca\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:25.223631 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.223635 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th45m\" (UniqueName: \"kubernetes.io/projected/4886fe04-08d5-4ceb-9963-0080a670be42-kube-api-access-th45m\") on node \"ip-10-0-130-239.ec2.internal\" DevicePath \"\"" Jul 06 20:02:25.260603 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.260571 2576 generic.go:356] "Generic (PLEG): container finished" podID="4886fe04-08d5-4ceb-9963-0080a670be42" containerID="9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f" exitCode=0 Jul 06 20:02:25.260733 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.260611 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" Jul 06 20:02:25.260733 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.260657 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" event={"ID":"4886fe04-08d5-4ceb-9963-0080a670be42","Type":"ContainerDied","Data":"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f"} Jul 06 20:02:25.260733 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.260705 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl" event={"ID":"4886fe04-08d5-4ceb-9963-0080a670be42","Type":"ContainerDied","Data":"5f192277ca879b5ee2eda0b30c24b93c32e1aaac6bcc4401a8ddb654913e7cae"} Jul 06 20:02:25.260733 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.260731 2576 scope.go:117] "RemoveContainer" containerID="9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f" Jul 06 20:02:25.270163 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.270122 2576 scope.go:117] "RemoveContainer" containerID="9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f" Jul 06 20:02:25.270403 ip-10-0-130-239 kubenswrapper[2576]: E0706 20:02:25.270384 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f\": container with ID starting with 9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f not found: ID does not exist" containerID="9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f" Jul 06 20:02:25.270474 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.270411 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f"} err="failed to get container status \"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f\": rpc error: code = NotFound desc = could not find container \"9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f\": container with ID starting with 9a5f36566a8691574fad137a17f8d289c128fc311fcedcd9765add19ce77925f not found: ID does not exist" Jul 06 20:02:25.282625 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.282598 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 20:02:25.288898 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.288869 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-97cecc-78fbc6c8cb-wc9sl"] Jul 06 20:02:25.850815 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:25.850783 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4886fe04-08d5-4ceb-9963-0080a670be42" path="/var/lib/kubelet/pods/4886fe04-08d5-4ceb-9963-0080a670be42/volumes" Jul 06 20:02:28.373728 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:28.373698 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-s8s82"] Jul 06 20:02:31.940624 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:31.940597 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-788c7f4f8f-6sxcn_c20c57bc-8267-4f52-8608-f2e773a1abc1/authorino/0.log" Jul 06 20:02:37.230929 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:37.230902 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gqb7x_b3052be5-f8ee-4084-a2ca-7b625b79f0f2/server/0.log" Jul 06 20:02:37.344868 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:37.344841 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r5vxj_7f818fd7-c436-4d9a-8d78-ffae230e063a/manager/0.log" Jul 06 20:02:37.577438 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:37.577363 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fd78d8d5-m5bn7_10149d48-7aa3-438d-ad38-a68e33be111d/copy-manifests/0.log" Jul 06 20:02:37.589564 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:37.589537 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fd78d8d5-m5bn7_10149d48-7aa3-438d-ad38-a68e33be111d/manager/0.log" Jul 06 20:02:38.580888 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.580859 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/util/0.log" Jul 06 20:02:38.587871 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.587849 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/pull/0.log" Jul 06 20:02:38.594595 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.594579 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/extract/0.log" Jul 06 20:02:38.704196 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.704170 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/extract/0.log" Jul 06 20:02:38.711226 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.711207 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/util/0.log" Jul 06 20:02:38.717917 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.717900 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/pull/0.log" Jul 06 20:02:38.830307 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.830279 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/util/0.log" Jul 06 20:02:38.836810 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.836749 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/pull/0.log" Jul 06 20:02:38.843460 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.843442 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/extract/0.log" Jul 06 20:02:38.952936 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.952915 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/util/0.log" Jul 06 20:02:38.959646 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.959617 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/pull/0.log" Jul 06 20:02:38.965853 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:38.965837 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/extract/0.log" Jul 06 20:02:39.081233 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:39.081211 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-788c7f4f8f-6sxcn_c20c57bc-8267-4f52-8608-f2e773a1abc1/authorino/0.log" Jul 06 20:02:39.766178 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:39.766150 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s8s82_0494c6aa-acfe-4fd0-8752-5d144bcb30c7/limitador/0.log" Jul 06 20:02:40.555744 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:40.555718 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc75879f5-btxg7_fea0f914-328b-4f48-a9db-65a93c985f1b/kube-auth-proxy/0.log" Jul 06 20:02:41.028107 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:41.028079 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-696bd7f7cc-8fbvx_0763d32a-7979-4dcb-8fff-d5842fa0ee83/router/0.log" Jul 06 20:02:45.877627 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877591 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sxl92/must-gather-rklzd"] Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877914 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" containerName="main" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877925 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" containerName="main" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877944 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97822741-fdd0-4be3-be40-4319db66c93c" containerName="maas-api" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877950 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="97822741-fdd0-4be3-be40-4319db66c93c" containerName="maas-api" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877955 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4886fe04-08d5-4ceb-9963-0080a670be42" containerName="maas-api" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.877961 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4886fe04-08d5-4ceb-9963-0080a670be42" containerName="maas-api" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.878021 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="97822741-fdd0-4be3-be40-4319db66c93c" containerName="maas-api" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.878031 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="98dbd9d3-6f61-4ac6-b024-fe95b7e7b8c0" containerName="main" Jul 06 20:02:45.878053 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.878039 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4886fe04-08d5-4ceb-9963-0080a670be42" containerName="maas-api" Jul 06 20:02:45.882638 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.882616 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:45.885975 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.885953 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sxl92\"/\"kube-root-ca.crt\"" Jul 06 20:02:45.886792 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.886772 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sxl92\"/\"default-dockercfg-jrspn\"" Jul 06 20:02:45.886792 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.886786 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sxl92\"/\"openshift-service-ca.crt\"" Jul 06 20:02:45.894887 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.894866 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxl92/must-gather-rklzd"] Jul 06 20:02:45.988346 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.988318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8a2068d-fad8-42b9-bfe1-666d77d16360-must-gather-output\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:45.988480 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:45.988355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qgfr\" (UniqueName: \"kubernetes.io/projected/c8a2068d-fad8-42b9-bfe1-666d77d16360-kube-api-access-2qgfr\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.089222 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.089187 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8a2068d-fad8-42b9-bfe1-666d77d16360-must-gather-output\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.089399 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.089233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qgfr\" (UniqueName: \"kubernetes.io/projected/c8a2068d-fad8-42b9-bfe1-666d77d16360-kube-api-access-2qgfr\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.089511 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.089491 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8a2068d-fad8-42b9-bfe1-666d77d16360-must-gather-output\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.098804 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.098775 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qgfr\" (UniqueName: \"kubernetes.io/projected/c8a2068d-fad8-42b9-bfe1-666d77d16360-kube-api-access-2qgfr\") pod \"must-gather-rklzd\" (UID: \"c8a2068d-fad8-42b9-bfe1-666d77d16360\") " pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.192216 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.192190 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxl92/must-gather-rklzd" Jul 06 20:02:46.324353 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.324329 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxl92/must-gather-rklzd"] Jul 06 20:02:46.326099 ip-10-0-130-239 kubenswrapper[2576]: W0706 20:02:46.326073 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8a2068d_fad8_42b9_bfe1_666d77d16360.slice/crio-f4574034279292cd3bff4d2375785df16f7f33b0159760f1684cb3502da2f695 WatchSource:0}: Error finding container f4574034279292cd3bff4d2375785df16f7f33b0159760f1684cb3502da2f695: Status 404 returned error can't find the container with id f4574034279292cd3bff4d2375785df16f7f33b0159760f1684cb3502da2f695 Jul 06 20:02:46.327914 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.327898 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:02:46.361667 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:46.361645 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxl92/must-gather-rklzd" event={"ID":"c8a2068d-fad8-42b9-bfe1-666d77d16360","Type":"ContainerStarted","Data":"f4574034279292cd3bff4d2375785df16f7f33b0159760f1684cb3502da2f695"} Jul 06 20:02:48.377204 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:48.377097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxl92/must-gather-rklzd" event={"ID":"c8a2068d-fad8-42b9-bfe1-666d77d16360","Type":"ContainerStarted","Data":"e031ef4c55b5b21b8b0fa7a8da8a1a3f1729da041b18ee553368c20516d2c0a7"} Jul 06 20:02:48.377204 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:48.377172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxl92/must-gather-rklzd" event={"ID":"c8a2068d-fad8-42b9-bfe1-666d77d16360","Type":"ContainerStarted","Data":"8f44fcad44638cad1d144984e09edfab1bf3a1093af42375d853f0df1c0936c6"} Jul 06 20:02:48.397244 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:48.397196 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sxl92/must-gather-rklzd" podStartSLOduration=1.9723118739999999 podStartE2EDuration="3.397180717s" totalImagesPullingTime="1.424868843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:02:48.395159212 +0000 UTC m=+2583.075193010" watchObservedRunningTime="2026-07-06 20:02:48.397180717 +0000 UTC m=+2583.077214493" Jul 06 20:02:49.443553 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:49.443489 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lr9nc_e69ee40d-db1c-4469-929c-9338f79a9e97/global-pull-secret-syncer/0.log" Jul 06 20:02:49.621727 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:49.621699 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w76qj_39112394-12d7-400c-a71c-921122fd5f0f/konnectivity-agent/0.log" Jul 06 20:02:49.643887 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:49.643858 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-239.ec2.internal_1115f28ae51a4317d8862e47dd2c12eb/haproxy/0.log" Jul 06 20:02:54.185049 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.185001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/extract/0.log" Jul 06 20:02:54.211853 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.211814 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/util/0.log" Jul 06 20:02:54.237795 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.237760 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f6lxl_55c9ef81-648c-428f-8e8e-f5e21f36ac27/pull/0.log" Jul 06 20:02:54.277459 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.277367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/extract/0.log" Jul 06 20:02:54.303519 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.303424 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/util/0.log" Jul 06 20:02:54.329586 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.329525 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0292d2_8b2e2c28-043f-48a0-8c64-d3a28efc960a/pull/0.log" Jul 06 20:02:54.359553 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.359526 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/extract/0.log" Jul 06 20:02:54.385888 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.385863 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/util/0.log" Jul 06 20:02:54.410777 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.410708 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tkhvc_76bff87b-1ee9-4f47-a743-c75ff7f2c2b3/pull/0.log" Jul 06 20:02:54.444904 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.444828 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/extract/0.log" Jul 06 20:02:54.475461 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.475436 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/util/0.log" Jul 06 20:02:54.502658 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.502568 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14dmgl_99d017f9-9690-4308-8bdb-dd20171413e9/pull/0.log" Jul 06 20:02:54.876858 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:54.876398 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-788c7f4f8f-6sxcn_c20c57bc-8267-4f52-8608-f2e773a1abc1/authorino/0.log" Jul 06 20:02:55.333224 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:55.333192 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-s8s82_0494c6aa-acfe-4fd0-8752-5d144bcb30c7/limitador/0.log" Jul 06 20:02:57.375344 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:57.375315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-4z89j_3581fad1-94cd-42ee-9aa7-abb78fa9cfec/cluster-monitoring-operator/0.log" Jul 06 20:02:57.555287 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:57.555254 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hxgz_051e7ed2-98de-4766-89bd-295206adf997/node-exporter/0.log" Jul 06 20:02:57.580806 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:57.580775 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hxgz_051e7ed2-98de-4766-89bd-295206adf997/kube-rbac-proxy/0.log" Jul 06 20:02:57.608865 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:57.608839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2hxgz_051e7ed2-98de-4766-89bd-295206adf997/init-textfile/0.log" Jul 06 20:02:58.175215 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:58.175127 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-gjh44_908a4923-8ce3-4024-9587-31b9851a4c6c/prometheus-operator-admission-webhook/0.log" Jul 06 20:02:59.736995 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.736941 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n"] Jul 06 20:02:59.743544 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.743520 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.757985 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.757962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n"] Jul 06 20:02:59.824913 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.824876 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-podres\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.825112 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.824945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-proc\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.825112 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.824970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-lib-modules\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.825112 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.824997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2bvx\" (UniqueName: \"kubernetes.io/projected/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-kube-api-access-c2bvx\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.825112 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.825077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-sys\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.926646 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.926601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-podres\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.926836 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.926717 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-proc\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.926836 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.926753 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-lib-modules\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.926836 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.926784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2bvx\" (UniqueName: \"kubernetes.io/projected/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-kube-api-access-c2bvx\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.927024 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.926908 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-sys\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.927024 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.927003 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-sys\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.927157 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.927127 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-podres\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.927939 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.927911 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-proc\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.928065 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.928046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-lib-modules\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:02:59.938676 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:02:59.938650 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2bvx\" (UniqueName: \"kubernetes.io/projected/d5eec5ca-2abb-40ae-8c15-a110a5e726bd-kube-api-access-c2bvx\") pod \"perf-node-gather-daemonset-7tt4n\" (UID: \"d5eec5ca-2abb-40ae-8c15-a110a5e726bd\") " pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:03:00.061257 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.061172 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:03:00.232265 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.232239 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n"] Jul 06 20:03:00.240574 ip-10-0-130-239 kubenswrapper[2576]: W0706 20:03:00.236675 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5eec5ca_2abb_40ae_8c15_a110a5e726bd.slice/crio-db517df2793cbe9c135fcfa31adf13468f1caed5915c5357d700a61430b64af1 WatchSource:0}: Error finding container db517df2793cbe9c135fcfa31adf13468f1caed5915c5357d700a61430b64af1: Status 404 returned error can't find the container with id db517df2793cbe9c135fcfa31adf13468f1caed5915c5357d700a61430b64af1 Jul 06 20:03:00.299336 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.299313 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/1.log" Jul 06 20:03:00.304794 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.304774 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pl7cl_408aa770-e6eb-4eb9-b635-8e65281f4183/console-operator/2.log" Jul 06 20:03:00.467200 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.467165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" event={"ID":"d5eec5ca-2abb-40ae-8c15-a110a5e726bd","Type":"ContainerStarted","Data":"cd14dd3e630a47816295008c902d84c5dd66da9b4bb21e47a81e94e296f075a5"} Jul 06 20:03:00.467380 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.467226 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" event={"ID":"d5eec5ca-2abb-40ae-8c15-a110a5e726bd","Type":"ContainerStarted","Data":"db517df2793cbe9c135fcfa31adf13468f1caed5915c5357d700a61430b64af1"} Jul 06 20:03:00.467443 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.467414 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:03:00.485088 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:00.485044 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" podStartSLOduration=1.485030717 podStartE2EDuration="1.485030717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:03:00.483976487 +0000 UTC m=+2595.164010284" watchObservedRunningTime="2026-07-06 20:03:00.485030717 +0000 UTC m=+2595.165064551" Jul 06 20:03:01.495338 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:01.495309 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-2hrr8_cd0e201a-7fcf-41b4-a2e3-49fae83fba8d/volume-data-source-validator/0.log" Jul 06 20:03:02.568501 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:02.568468 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n97st_d7679d0b-cbec-44fd-b860-7c9337800263/dns/0.log" Jul 06 20:03:02.589192 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:02.589164 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n97st_d7679d0b-cbec-44fd-b860-7c9337800263/kube-rbac-proxy/0.log" Jul 06 20:03:02.711630 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:02.711602 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvh5p_cf9bedb2-db52-4c8a-96f3-2834416a1c76/dns-node-resolver/0.log" Jul 06 20:03:03.294752 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:03.294713 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5rc7_b7e4997e-01f4-4909-a2a6-3eee839cf20b/node-ca/0.log" Jul 06 20:03:04.701028 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:04.701003 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc75879f5-btxg7_fea0f914-328b-4f48-a9db-65a93c985f1b/kube-auth-proxy/0.log" Jul 06 20:03:04.927322 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:04.927291 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-696bd7f7cc-8fbvx_0763d32a-7979-4dcb-8fff-d5842fa0ee83/router/0.log" Jul 06 20:03:05.720688 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:05.720653 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q4dpp_b4e96682-a1fb-4d21-b66e-f82a843dc66c/serve-healthcheck-canary/0.log" Jul 06 20:03:06.356599 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.356567 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-gdrrs_9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65/insights-operator/1.log" Jul 06 20:03:06.357534 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.357511 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-gdrrs_9abd5c3a-1ca0-4a9c-8a52-2a428fd2af65/insights-operator/0.log" Jul 06 20:03:06.382270 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.382246 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nz4z_0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26/kube-rbac-proxy/0.log" Jul 06 20:03:06.407416 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.407389 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nz4z_0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26/exporter/0.log" Jul 06 20:03:06.432262 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.432240 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nz4z_0aaf41ad-67fc-4fbc-ae11-af6a5ae0ab26/extractor/0.log" Jul 06 20:03:06.483126 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:06.483103 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sxl92/perf-node-gather-daemonset-7tt4n" Jul 06 20:03:09.496833 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:09.496805 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gqb7x_b3052be5-f8ee-4084-a2ca-7b625b79f0f2/server/0.log" Jul 06 20:03:09.598930 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:09.597879 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r5vxj_7f818fd7-c436-4d9a-8d78-ffae230e063a/manager/0.log" Jul 06 20:03:09.749712 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:09.749679 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fd78d8d5-m5bn7_10149d48-7aa3-438d-ad38-a68e33be111d/manager/0.log" Jul 06 20:03:09.788014 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:09.787996 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fd78d8d5-m5bn7_10149d48-7aa3-438d-ad38-a68e33be111d/copy-manifests/0.log" Jul 06 20:03:09.969460 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:09.969431 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc75879f5-btxg7_fea0f914-328b-4f48-a9db-65a93c985f1b/kube-auth-proxy/0.log" Jul 06 20:03:10.173401 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:10.173367 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-696bd7f7cc-8fbvx_0763d32a-7979-4dcb-8fff-d5842fa0ee83/router/0.log" Jul 06 20:03:11.232607 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:11.232575 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d885dd8f9-t2vmz_f279d084-267b-4a33-a17e-274bc97545ab/manager/0.log" Jul 06 20:03:11.579361 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:11.579333 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bc75879f5-btxg7_fea0f914-328b-4f48-a9db-65a93c985f1b/kube-auth-proxy/0.log" Jul 06 20:03:11.768875 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:11.768800 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-696bd7f7cc-8fbvx_0763d32a-7979-4dcb-8fff-d5842fa0ee83/router/0.log" Jul 06 20:03:16.499050 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:16.499017 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wm6gc_f44fb294-1867-4650-a8c8-1a7d7cf0378b/kube-storage-version-migrator-operator/1.log" Jul 06 20:03:16.499902 ip-10-0-130-239 kubenswrapper[2576]: I0706 20:03:16.499884 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wm6gc_f44fb294-1867-4650-a8c8-1a7d7cf0378b/kube-storage-version-migrator-operator/0.log"