Jul 18 11:47:09.858669 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 18 11:47:09.858681 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 18 11:47:09.858688 ip-10-0-130-242 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 18 11:47:09.858933 ip-10-0-130-242 systemd[1]: Failed to start Kubernetes Kubelet. Jul 18 11:47:20.107947 ip-10-0-130-242 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 18 11:47:20.107962 ip-10-0-130-242 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3bc73c765269428481a3e5687606bf93 -- Jul 18 11:48:49.531229 ip-10-0-130-242 systemd[1]: Starting Kubernetes Kubelet... Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.922446 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925878 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925889 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925893 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925896 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925899 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925902 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925905 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:49.983763 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925907 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925911 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925913 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925918 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925921 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925923 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925926 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925929 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925931 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925934 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925938 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925940 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925943 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925946 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925950 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925970 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925973 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925976 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925979 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:49.985790 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925981 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925984 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925987 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925989 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925992 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925995 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.925997 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926000 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926002 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926005 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926008 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926010 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926013 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926015 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926018 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926020 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926023 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926026 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926028 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926031 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:49.988013 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926033 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926036 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926038 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926041 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926043 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926046 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926049 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926053 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926058 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926062 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926064 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926067 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926070 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926073 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926076 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926078 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926081 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926087 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926089 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926092 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.003627 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926094 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.000563 ip-10-0-130-242 systemd[1]: Started Kubernetes Kubelet. Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926099 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926103 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926107 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926110 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926113 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926116 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926119 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926123 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926125 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926129 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926134 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926136 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926139 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926141 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926144 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926147 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926151 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926153 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926156 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.006196 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926159 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926162 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926174 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926177 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926180 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926182 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926185 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926188 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926191 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926194 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926198 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926203 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926206 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926895 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926902 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926905 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926908 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926910 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926913 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926916 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.016665 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926918 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926921 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926924 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926927 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926929 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926932 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926934 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926936 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926939 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926941 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926944 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926946 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926949 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926952 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926969 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926972 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926975 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926978 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926981 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926984 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:50.020230 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926986 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926991 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926994 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.926997 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927000 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927002 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927005 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927008 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927010 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927013 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927016 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927019 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927022 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927024 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927027 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927029 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927032 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927034 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927037 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927040 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.035733 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927044 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927047 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927049 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927052 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927054 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927060 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927064 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927068 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927072 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927074 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927077 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927080 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927082 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927086 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927089 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927091 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927094 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927097 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927100 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:50.038394 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927103 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927105 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927108 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927110 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927113 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927116 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927119 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927121 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927124 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927127 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927131 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927134 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927137 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927140 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927143 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927146 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927148 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927151 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927156 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:50.051608 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927159 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927162 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927164 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927167 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927169 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927172 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927174 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927179 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927181 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927184 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927186 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927188 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927191 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927193 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927544 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927548 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927551 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927554 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927556 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927559 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.054640 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927562 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927565 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927567 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927569 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927572 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927575 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927578 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927582 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927584 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927587 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927589 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927593 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927596 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927599 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927601 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927604 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927607 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927609 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927611 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927615 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.067894 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927618 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927620 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927623 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927625 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927628 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927630 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927633 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927635 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927638 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927640 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927643 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927645 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927648 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927650 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927653 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927655 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927658 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927660 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927663 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927665 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.094253 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927668 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927670 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927673 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927677 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927680 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927683 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927686 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927688 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927691 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927694 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927696 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927700 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927702 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927704 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927707 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927710 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927712 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927715 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927717 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:50.218760 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927720 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927722 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927724 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927727 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927729 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927732 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927736 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927740 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927742 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927745 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927748 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927750 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927753 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927755 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927758 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927760 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927764 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927766 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927769 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927772 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:50.219529 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927775 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927778 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927780 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927784 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927789 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927792 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927795 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927797 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927800 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927803 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927805 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927808 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927810 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.927813 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927882 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927894 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927899 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927905 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927909 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927912 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 11:48:50.220175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927916 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927920 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927923 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927926 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927930 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927933 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927936 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927939 2578 flags.go:64] FLAG: --cgroup-root="" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927943 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927946 2578 flags.go:64] FLAG: --client-ca-file="" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927949 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927967 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927971 2578 flags.go:64] FLAG: --cluster-domain="" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927974 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927978 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927981 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927986 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927989 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927992 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927996 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.927999 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928003 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928005 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928008 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 11:48:50.220836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928011 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928014 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928018 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928021 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928024 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928026 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928029 2578 flags.go:64] FLAG: --enable-server="true" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928032 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928036 2578 flags.go:64] FLAG: --event-burst="100" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928039 2578 flags.go:64] FLAG: --event-qps="50" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928042 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928045 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928048 2578 flags.go:64] FLAG: --eviction-hard="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928052 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928055 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928058 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928063 2578 flags.go:64] FLAG: --eviction-soft="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928066 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928070 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928073 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928076 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928078 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928082 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928085 2578 flags.go:64] FLAG: --feature-gates="" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928090 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 11:48:50.221644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928093 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928096 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928099 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928102 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928105 2578 flags.go:64] FLAG: --help="false" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928108 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928111 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928114 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928117 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928121 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928124 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928127 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928130 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928133 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928135 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928138 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928141 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928144 2578 flags.go:64] FLAG: --kube-reserved="" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928147 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928150 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928153 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928155 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928158 2578 flags.go:64] FLAG: --lock-file="" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928162 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 11:48:50.222519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928165 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928168 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928173 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928176 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928179 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928182 2578 flags.go:64] FLAG: --logging-format="text" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928185 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928189 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928192 2578 flags.go:64] FLAG: --manifest-url="" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928195 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928199 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928202 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928206 2578 flags.go:64] FLAG: --max-pods="110" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928209 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928212 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928214 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928217 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928220 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928223 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928226 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928232 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928235 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928238 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 11:48:50.225664 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928241 2578 flags.go:64] FLAG: --pod-cidr="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928244 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928249 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928252 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928254 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928257 2578 flags.go:64] FLAG: --port="10250" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928260 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928263 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a602bfc6a111d959" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928266 2578 flags.go:64] FLAG: --qos-reserved="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928270 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928273 2578 flags.go:64] FLAG: --register-node="true" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928276 2578 flags.go:64] FLAG: --register-with-taints="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928280 2578 flags.go:64] FLAG: --registry-burst="10" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928283 2578 flags.go:64] FLAG: --registry-qps="5" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928285 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928288 2578 flags.go:64] FLAG: --reserved-memory="" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928293 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928296 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928300 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928302 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928305 2578 flags.go:64] FLAG: --runonce="false" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928308 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928310 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928313 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928316 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 11:48:50.226486 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928319 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928322 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928325 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928328 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928331 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928334 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928336 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928339 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928342 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928345 2578 flags.go:64] FLAG: --system-cgroups="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928348 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928353 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928355 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928358 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928361 2578 flags.go:64] FLAG: --tls-min-version="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928364 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928368 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928371 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928374 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928377 2578 flags.go:64] FLAG: --v="2" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928380 2578 flags.go:64] FLAG: --version="false" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928384 2578 flags.go:64] FLAG: --vmodule="" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928388 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928392 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928537 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.229788 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928542 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928546 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928549 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928551 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928554 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928557 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928560 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928562 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928565 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928567 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928570 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928573 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928575 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928578 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928580 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928583 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928586 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928589 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928591 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928594 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:50.230566 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928596 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928599 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928601 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928605 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928608 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928610 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928613 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928617 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928620 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928623 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928627 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928630 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928638 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928641 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928644 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928647 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928651 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928654 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928656 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928659 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:50.231138 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928662 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928664 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928666 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928669 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928671 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928674 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928676 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928682 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928685 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928687 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928690 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928693 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928695 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928698 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928700 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928705 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928709 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928711 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928714 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928716 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.231727 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928719 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928721 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928725 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928727 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928730 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928738 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928741 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928743 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928746 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928748 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928750 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928753 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928755 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928758 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928760 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928763 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928765 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928773 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928776 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:50.232487 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928779 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928781 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928783 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928786 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928789 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928791 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928794 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928796 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928800 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928803 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928805 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928808 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928810 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928813 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928815 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928819 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928821 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928824 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:50.383078 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.928826 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.928837 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.937875 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.938021 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.938807 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.938814 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938872 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938879 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938883 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938886 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938889 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938892 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938895 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938899 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.383783 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938901 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938904 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938907 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938910 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938912 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938915 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938918 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938920 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938923 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938926 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938928 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938931 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938934 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938936 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938939 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938941 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938944 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938947 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938950 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938970 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.384398 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938978 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938981 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938983 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938986 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938988 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938991 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938994 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.938996 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939000 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939004 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939007 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939010 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939013 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939015 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939018 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939021 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939023 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939026 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939028 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939031 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.385213 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939034 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939036 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939039 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939041 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939044 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939047 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939050 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939052 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939055 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939058 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939062 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939065 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939069 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939072 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939075 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939078 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939081 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939084 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939086 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:50.385821 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939089 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939092 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939095 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939098 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939101 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939104 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939106 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939109 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939112 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939114 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939117 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939119 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939123 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939125 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939128 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939130 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939133 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939135 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939138 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939140 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.386458 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939143 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939145 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939148 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939150 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939153 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939157 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939159 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939162 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939164 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939167 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939169 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939172 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.939177 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939297 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939303 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 11:48:50.387062 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939305 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939309 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939312 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939315 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939317 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939320 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939323 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939325 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939329 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939333 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939336 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939339 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939341 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939344 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939347 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939349 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939352 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939354 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939357 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 11:48:50.387485 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939359 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939362 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939364 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939367 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939371 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939374 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939376 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939379 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939381 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939384 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939386 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939389 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939392 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939394 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939398 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939401 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939403 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939406 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939409 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939411 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 11:48:50.388035 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939414 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939416 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939419 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939421 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939424 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939427 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939430 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939432 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939434 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939437 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939439 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939441 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939444 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939446 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939449 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939451 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939455 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939458 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939460 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939463 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 11:48:50.388564 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939465 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939468 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939470 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939473 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939475 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939478 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939481 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939484 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939487 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939489 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939492 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939494 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939498 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939501 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939504 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939506 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939509 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939513 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939516 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 11:48:50.389154 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939518 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939521 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939523 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939526 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939528 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939531 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939533 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939535 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939538 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939542 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939545 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939547 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939550 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939552 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939555 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939558 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939560 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939563 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 11:48:50.389659 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:49.939566 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.939573 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.939698 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.941779 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.943022 2578 server.go:1019] "Starting client certificate rotation" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.943121 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.943158 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.945399 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.946518 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.963587 2578 log.go:25] "Validated CRI v1 runtime API" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.966524 2578 log.go:25] "Validated CRI v1 image API" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.969874 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 11:48:50.390186 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.972744 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cff0e996-41d8-4cab-9155-4fc243f85b45:/dev/nvme0n1p4 e4b5ea10-3dcc-465e-b039-c096cd7a6394:/dev/nvme0n1p3] Jul 18 11:48:50.390538 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.972758 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 11:48:50.390538 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.974835 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.979481 2578 manager.go:216] Machine: {Timestamp:2026-07-18 11:48:49.978252791 +0000 UTC m=+0.329402328 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108271 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29d8f27ff0543b209410677d5c14e3 SystemUUID:ec29d8f2-7ff0-543b-2094-10677d5c14e3 BootID:3bc73c76-5269-4284-81a3-e5687606bf93 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:88:80:85:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:88:80:85:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:88:4d:55:d3:76 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.979583 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.979691 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.982881 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.982917 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-242.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.983681 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.983692 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.983700 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.984337 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.985576 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989688 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989797 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989824 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989834 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989849 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989859 2578 kubelet.go:398] "Adding apiserver pod source" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.989868 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 11:48:50.391002 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.990268 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.990291 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.992252 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.993619 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.993645 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994252 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994266 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994272 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994278 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994286 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994292 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994316 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994322 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994329 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994335 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994343 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.994351 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.996041 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.996054 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.999566 2578 server.go:1268] "Started kubelet" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.999637 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.999648 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:49.999780 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.001169 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-242.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.001204 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.001658 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.001657 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.002387 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.004758 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.005576 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-242.ec2.internal.18c35fe89a97ccb7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-242.ec2.internal,UID:ip-10-0-130-242.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-242.ec2.internal,},FirstTimestamp:2026-07-18 11:48:49.999539383 +0000 UTC m=+0.350688921,LastTimestamp:2026-07-18 11:48:49.999539383 +0000 UTC m=+0.350688921,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-242.ec2.internal,}" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.008920 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.008933 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.009041 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009514 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009534 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009517 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009674 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009682 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009836 2578 factory.go:55] Registering systemd factory Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.009853 2578 factory.go:223] Registration of the systemd container factory successfully Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010087 2578 factory.go:153] Registering CRI-O factory Jul 18 11:48:50.391567 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010097 2578 factory.go:223] Registration of the crio container factory successfully Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010137 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010155 2578 factory.go:103] Registering Raw factory Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010166 2578 manager.go:1196] Started watching for new ooms in manager Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.010600 2578 manager.go:349] Starting recovery of all containers Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.013015 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.013132 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.018047 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzgjg" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.022841 2578 manager.go:354] Recovery completed Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.024939 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzgjg" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.027400 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029177 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029209 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029219 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029717 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029732 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029746 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.029722 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.030408 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.030430 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.030440 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.031685 2578 policy_none.go:49] "None policy: Start" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.031698 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.031708 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.030856 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-242.ec2.internal.18c35fe89c5c41cc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-242.ec2.internal,UID:ip-10-0-130-242.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-242.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-242.ec2.internal,},FirstTimestamp:2026-07-18 11:48:50.029191628 +0000 UTC m=+0.380341166,LastTimestamp:2026-07-18 11:48:50.029191628 +0000 UTC m=+0.380341166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-242.ec2.internal,}" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.033674 2578 policy_none.go:47] "Start" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.104599 2578 manager.go:337] "Starting Device Plugin manager" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.104627 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.104636 2578 server.go:87] "Starting device plugin registration server" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.104870 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.104880 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.104987 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.105084 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.105094 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.105726 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.105767 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.125060 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.126384 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.126426 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 11:48:50.530872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.126448 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.126488 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.128746 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.205792 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.206810 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.206848 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.206863 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.206892 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.218610 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.218633 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-242.ec2.internal\": node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.227516 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal"] Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.227577 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.229454 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.229483 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.229496 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.231702 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.231814 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.231841 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232394 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232415 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232428 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232459 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232480 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.232495 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.234551 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.234572 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.235217 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientMemory" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.235238 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.235251 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeHasSufficientPID" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.240851 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.265555 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-242.ec2.internal\" not found" node="ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.269796 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-242.ec2.internal\" not found" node="ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.311719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532018 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.311750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.311767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcb4fafed7c79e7e7353434bd1881be-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"5dcb4fafed7c79e7e7353434bd1881be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.341449 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcb4fafed7c79e7e7353434bd1881be-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"5dcb4fafed7c79e7e7353434bd1881be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412912 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412923 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5797e5b85abf8a89c911a3546660467-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal\" (UID: \"f5797e5b85abf8a89c911a3546660467\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.412947 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcb4fafed7c79e7e7353434bd1881be-config\") pod \"kube-apiserver-proxy-ip-10-0-130-242.ec2.internal\" (UID: \"5dcb4fafed7c79e7e7353434bd1881be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.532894 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.441928 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.648857 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.542715 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.648857 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.567875 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.648857 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.572744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:50.648857 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.643218 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:50.781255 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.744407 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:50.845190 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-242.ec2.internal\" not found" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:50.925080 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.209090 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.222144 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.222257 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.222290 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 11:48:52.233359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.222292 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.269092 2578 apiserver.go:52] "Watching apiserver" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.282488 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.282797 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-wh7n8","openshift-network-diagnostics/network-check-target-998xh","openshift-network-operator/iptables-alerter-wppmv","kube-system/global-pull-secret-syncer-ndsm5","kube-system/konnectivity-agent-7c5sk","openshift-cluster-node-tuning-operator/tuned-2l7rc","openshift-dns/node-resolver-jt2z9","openshift-image-registry/node-ca-49tjp","openshift-multus/multus-additional-cni-plugins-srfjb","openshift-ovn-kubernetes/ovnkube-node-tqdw9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq","openshift-multus/multus-5v5jq"] Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.285506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.285568 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.288125 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.288699 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.288911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.288982 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.291118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.293074 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.293078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-26bjq\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.293148 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.293263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.293452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.293495 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.296589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.296867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7vsq\" (UniqueName: \"kubernetes.io/projected/e785910e-4713-4292-acc8-d9d884f9d8db-kube-api-access-c7vsq\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.297516 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.298025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.298612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.300761 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.300948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301031 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301057 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301223 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 11:48:52.407991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301395 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-748rq\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301515 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.301748 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g6gg9\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.303098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.303144 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.303329 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m2rxv\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.303355 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.303489 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.304762 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305261 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l6jcv\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305662 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305895 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305942 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.305979 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308394 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308693 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 11:43:51 +0000 UTC" deadline="2028-02-21 07:51:29.600021265 +0000 UTC" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308723 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308730 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13988h2m37.291294396s" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308731 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308749 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l6wq9\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.308797 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.309013 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.309521 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.311338 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.311556 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.311898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.311920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.312218 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.312340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5pnxg\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.312549 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.313174 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.314984 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal"] Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315031 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315359 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal"] Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315440 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 11:48:52.413416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315460 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g7rnt\"" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.315546 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.319923 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.320195 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.320282 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kdktk\"" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.327500 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddvcx" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.333519 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddvcx" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.389692 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-registration-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-etc-selinux\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-k8s-cni-cncf-io\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vts7t\" (UniqueName: \"kubernetes.io/projected/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-kube-api-access-vts7t\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysconfig\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-kubernetes\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85bdf996-67e4-41f3-95fb-798cc6c89502-ovn-node-metrics-cert\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-host\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-tmp\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397926 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbh6\" (UniqueName: \"kubernetes.io/projected/5dd3d632-53fe-4f73-b152-c10013ca4bc4-kube-api-access-5jbh6\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-systemd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-modprobe-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.397989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7vsq\" (UniqueName: \"kubernetes.io/projected/e785910e-4713-4292-acc8-d9d884f9d8db-kube-api-access-c7vsq\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398071 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-kubelet-config\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-dbus\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-slash\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-script-lib\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-socket-dir-parent\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cnibin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-netns\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgks6\" (UniqueName: \"kubernetes.io/projected/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-kube-api-access-bgks6\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-log-socket\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.415882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1e7deda4-422f-4caa-93f0-876b65fdb720-hosts-file\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398523 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-system-cni-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-os-release\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-binary-copy\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfdbz\" (UniqueName: \"kubernetes.io/projected/03b66d23-e64b-4076-9494-c10d0b7a891d-kube-api-access-tfdbz\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.398738 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-etc-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.398817 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:48:52.898788124 +0000 UTC m=+1.970847619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-config\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-os-release\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398866 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-konnectivity-ca\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl5ms\" (UniqueName: \"kubernetes.io/projected/1e7deda4-422f-4caa-93f0-876b65fdb720-kube-api-access-hl5ms\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs7s5\" (UniqueName: \"kubernetes.io/projected/a5fc1488-82a5-4f0a-a201-4d614587a448-kube-api-access-bs7s5\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.398981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-systemd\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-conf\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.416495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-var-lib-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-env-overrides\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-device-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-bin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-netd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-multus\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-kubelet\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-multus-certs\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65fca35e-cf89-44f8-ac27-6d26f9b1753a-host\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65fca35e-cf89-44f8-ac27-6d26f9b1753a-serviceca\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2w5p\" (UniqueName: \"kubernetes.io/projected/65fca35e-cf89-44f8-ac27-6d26f9b1753a-kube-api-access-l2w5p\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-socket-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-sys-fs\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cni-binary-copy\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-hostroot\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-agent-certs\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.417083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-run\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-lib-modules\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-system-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-iptables-alerter-script\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.399997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d6dp\" (UniqueName: \"kubernetes.io/projected/85bdf996-67e4-41f3-95fb-798cc6c89502-kube-api-access-7d6dp\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400019 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-sys\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-ovn\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-conf-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-kubelet\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-systemd-units\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-daemon-config\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-etc-kubernetes\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.417812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-bin\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-host-slash\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e7deda4-422f-4caa-93f0-876b65fdb720-tmp-dir\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-var-lib-kubelet\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-node-log\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-cnibin\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-tuned\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.400286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-netns\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.404800 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.404817 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.404829 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.404940 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:52.904922942 +0000 UTC m=+1.976982432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.405649 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 11:48:52.418448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.409973 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7vsq\" (UniqueName: \"kubernetes.io/projected/e785910e-4713-4292-acc8-d9d884f9d8db-kube-api-access-c7vsq\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509141 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-systemd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-modprobe-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-kubelet-config\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509253 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-kubelet-config\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509255 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-systemd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-modprobe-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-dbus\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-slash\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-script-lib\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c73af55a-7e9a-416b-8144-38477c907ba0-dbus\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-socket-dir-parent\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-slash\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509457 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cnibin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509445 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.527867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-netns\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509450 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cnibin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-socket-dir-parent\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgks6\" (UniqueName: \"kubernetes.io/projected/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-kube-api-access-bgks6\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509529 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-netns\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-log-socket\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1e7deda4-422f-4caa-93f0-876b65fdb720-hosts-file\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-system-cni-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-os-release\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1e7deda4-422f-4caa-93f0-876b65fdb720-hosts-file\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509604 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-log-socket\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-binary-copy\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-system-cni-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfdbz\" (UniqueName: \"kubernetes.io/projected/03b66d23-e64b-4076-9494-c10d0b7a891d-kube-api-access-tfdbz\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509750 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-etc-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.528983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-config\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-os-release\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-os-release\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-etc-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-konnectivity-ca\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl5ms\" (UniqueName: \"kubernetes.io/projected/1e7deda4-422f-4caa-93f0-876b65fdb720-kube-api-access-hl5ms\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs7s5\" (UniqueName: \"kubernetes.io/projected/a5fc1488-82a5-4f0a-a201-4d614587a448-kube-api-access-bs7s5\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-systemd\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-conf\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-var-lib-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-env-overrides\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-device-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-bin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.509976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-netd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510027 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-multus\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-kubelet\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.530464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-multus-certs\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-script-lib\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-multus-certs\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510177 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-ovn-kubernetes\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510189 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-netd\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510209 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-conf\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-kubelet\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-device-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510283 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-multus\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-os-release\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-var-lib-cni-bin\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-konnectivity-ca\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510368 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-var-lib-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65fca35e-cf89-44f8-ac27-6d26f9b1753a-host\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65fca35e-cf89-44f8-ac27-6d26f9b1753a-serviceca\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65fca35e-cf89-44f8-ac27-6d26f9b1753a-host\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-ovnkube-config\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2w5p\" (UniqueName: \"kubernetes.io/projected/65fca35e-cf89-44f8-ac27-6d26f9b1753a-kube-api-access-l2w5p\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.531344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510466 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85bdf996-67e4-41f3-95fb-798cc6c89502-env-overrides\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-socket-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-sys-fs\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510496 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-systemd\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cni-binary-copy\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-hostroot\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-agent-certs\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-hostroot\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-sys-fs\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-run\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510730 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-socket-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-lib-modules\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-system-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-iptables-alerter-script\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-run\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510825 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65fca35e-cf89-44f8-ac27-6d26f9b1753a-serviceca\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510838 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-lib-modules\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-binary-copy\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.532572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510896 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-system-cni-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.511019 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511018 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-cni-binary-copy\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.510999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d6dp\" (UniqueName: \"kubernetes.io/projected/85bdf996-67e4-41f3-95fb-798cc6c89502-kube-api-access-7d6dp\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-sys\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-ovn\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-openvswitch\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.511148 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:53.011128859 +0000 UTC m=+2.083188344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysctl-d\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-conf-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511160 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-run-ovn\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.533204 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511213 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-sys\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-conf-dir\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-kubelet\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-systemd-units\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-daemon-config\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511315 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-kubelet\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-etc-kubernetes\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511344 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-iptables-alerter-script\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-systemd-units\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-etc-kubernetes\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-bin\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-cni-bin\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-host-slash\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e7deda4-422f-4caa-93f0-876b65fdb720-tmp-dir\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-var-lib-kubelet\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511477 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-node-log\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-cnibin\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-tuned\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.536783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-netns\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-var-lib-kubelet\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03b66d23-e64b-4076-9494-c10d0b7a891d-cnibin\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-registration-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-node-log\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-etc-selinux\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-k8s-cni-cncf-io\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511655 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vts7t\" (UniqueName: \"kubernetes.io/projected/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-kube-api-access-vts7t\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysconfig\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511716 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-kubernetes\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511722 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e7deda4-422f-4caa-93f0-876b65fdb720-tmp-dir\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85bdf996-67e4-41f3-95fb-798cc6c89502-ovn-node-metrics-cert\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511786 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85bdf996-67e4-41f3-95fb-798cc6c89502-host-run-netns\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-host\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.537402 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-tmp\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-registration-dir\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511446 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-host-slash\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-multus-daemon-config\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbh6\" (UniqueName: \"kubernetes.io/projected/5dd3d632-53fe-4f73-b152-c10013ca4bc4-kube-api-access-5jbh6\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.511957 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-host\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.512004 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a5fc1488-82a5-4f0a-a201-4d614587a448-etc-selinux\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.512021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-host-run-k8s-cni-cncf-io\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.512071 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-kubernetes\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.512220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-sysconfig\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.512702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/03b66d23-e64b-4076-9494-c10d0b7a891d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.514403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-tmp\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.514506 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5dd3d632-53fe-4f73-b152-c10013ca4bc4-etc-tuned\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.514796 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5797e5b85abf8a89c911a3546660467.slice/crio-b61e4c66dfd54fa02bafdec64c4b92bded094c1d668dc0bd77acce976d5518c3 WatchSource:0}: Error finding container b61e4c66dfd54fa02bafdec64c4b92bded094c1d668dc0bd77acce976d5518c3: Status 404 returned error can't find the container with id b61e4c66dfd54fa02bafdec64c4b92bded094c1d668dc0bd77acce976d5518c3 Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.515277 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcb4fafed7c79e7e7353434bd1881be.slice/crio-b9db0803e72f53b262b36f3355bd029833cae46b640e8d3c9d7aec57da3331eb WatchSource:0}: Error finding container b9db0803e72f53b262b36f3355bd029833cae46b640e8d3c9d7aec57da3331eb: Status 404 returned error can't find the container with id b9db0803e72f53b262b36f3355bd029833cae46b640e8d3c9d7aec57da3331eb Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.516385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0b12eaf-f8e1-4431-9f83-c6ecb863b32c-agent-certs\") pod \"konnectivity-agent-7c5sk\" (UID: \"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c\") " pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.538279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.517850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgks6\" (UniqueName: \"kubernetes.io/projected/cc65839f-dcf6-4f4e-8320-acb7d2dde8be-kube-api-access-bgks6\") pod \"iptables-alerter-wppmv\" (UID: \"cc65839f-dcf6-4f4e-8320-acb7d2dde8be\") " pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.518384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfdbz\" (UniqueName: \"kubernetes.io/projected/03b66d23-e64b-4076-9494-c10d0b7a891d-kube-api-access-tfdbz\") pod \"multus-additional-cni-plugins-srfjb\" (UID: \"03b66d23-e64b-4076-9494-c10d0b7a891d\") " pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.518430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl5ms\" (UniqueName: \"kubernetes.io/projected/1e7deda4-422f-4caa-93f0-876b65fdb720-kube-api-access-hl5ms\") pod \"node-resolver-jt2z9\" (UID: \"1e7deda4-422f-4caa-93f0-876b65fdb720\") " pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.519688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2w5p\" (UniqueName: \"kubernetes.io/projected/65fca35e-cf89-44f8-ac27-6d26f9b1753a-kube-api-access-l2w5p\") pod \"node-ca-49tjp\" (UID: \"65fca35e-cf89-44f8-ac27-6d26f9b1753a\") " pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.519709 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs7s5\" (UniqueName: \"kubernetes.io/projected/a5fc1488-82a5-4f0a-a201-4d614587a448-kube-api-access-bs7s5\") pod \"aws-ebs-csi-driver-node-sq2mq\" (UID: \"a5fc1488-82a5-4f0a-a201-4d614587a448\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.520168 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbh6\" (UniqueName: \"kubernetes.io/projected/5dd3d632-53fe-4f73-b152-c10013ca4bc4-kube-api-access-5jbh6\") pod \"tuned-2l7rc\" (UID: \"5dd3d632-53fe-4f73-b152-c10013ca4bc4\") " pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.520771 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d6dp\" (UniqueName: \"kubernetes.io/projected/85bdf996-67e4-41f3-95fb-798cc6c89502-kube-api-access-7d6dp\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.520804 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.520901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vts7t\" (UniqueName: \"kubernetes.io/projected/0fe06274-ed0d-4b45-b177-61cd75f4ad1f-kube-api-access-vts7t\") pod \"multus-5v5jq\" (UID: \"0fe06274-ed0d-4b45-b177-61cd75f4ad1f\") " pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.538763 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.527868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85bdf996-67e4-41f3-95fb-798cc6c89502-ovn-node-metrics-cert\") pod \"ovnkube-node-tqdw9\" (UID: \"85bdf996-67e4-41f3-95fb-798cc6c89502\") " pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.621156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wppmv" Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.635409 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc65839f_dcf6_4f4e_8320_acb7d2dde8be.slice/crio-a996d02fdd49c964b7a853d10795ca6751e8c1b5dce4207ecb31566deb803aa6 WatchSource:0}: Error finding container a996d02fdd49c964b7a853d10795ca6751e8c1b5dce4207ecb31566deb803aa6: Status 404 returned error can't find the container with id a996d02fdd49c964b7a853d10795ca6751e8c1b5dce4207ecb31566deb803aa6 Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.645504 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.653960 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b12eaf_f8e1_4431_9f83_c6ecb863b32c.slice/crio-6c89f27f15507d01e37601fb3a02d64e7eb67247d8786ce24f07b24b62234b25 WatchSource:0}: Error finding container 6c89f27f15507d01e37601fb3a02d64e7eb67247d8786ce24f07b24b62234b25: Status 404 returned error can't find the container with id 6c89f27f15507d01e37601fb3a02d64e7eb67247d8786ce24f07b24b62234b25 Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.666788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" Jul 18 11:48:52.682462 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.674601 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd3d632_53fe_4f73_b152_c10013ca4bc4.slice/crio-897acb817724d8d76da9fd9a289a55d6889bdd8229ba96e0bea2a40ed3226a79 WatchSource:0}: Error finding container 897acb817724d8d76da9fd9a289a55d6889bdd8229ba96e0bea2a40ed3226a79: Status 404 returned error can't find the container with id 897acb817724d8d76da9fd9a289a55d6889bdd8229ba96e0bea2a40ed3226a79 Jul 18 11:48:52.686740 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.683251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jt2z9" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.693251 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7deda4_422f_4caa_93f0_876b65fdb720.slice/crio-1d317cf60afb672d17108d69c24d79b2f677a5165b8fd57fb7fd173943ec4cd4 WatchSource:0}: Error finding container 1d317cf60afb672d17108d69c24d79b2f677a5165b8fd57fb7fd173943ec4cd4: Status 404 returned error can't find the container with id 1d317cf60afb672d17108d69c24d79b2f677a5165b8fd57fb7fd173943ec4cd4 Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.705023 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-49tjp" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.712625 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65fca35e_cf89_44f8_ac27_6d26f9b1753a.slice/crio-72b69ca97e5d62a1d265b600e5875a0d6e08966a1233feef90f928e467584599 WatchSource:0}: Error finding container 72b69ca97e5d62a1d265b600e5875a0d6e08966a1233feef90f928e467584599: Status 404 returned error can't find the container with id 72b69ca97e5d62a1d265b600e5875a0d6e08966a1233feef90f928e467584599 Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.732255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-srfjb" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.751687 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.751928 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b66d23_e64b_4076_9494_c10d0b7a891d.slice/crio-09645d52b34e335da8c417d1715c28c0fbe03b16d3551f980452f40f25454aed WatchSource:0}: Error finding container 09645d52b34e335da8c417d1715c28c0fbe03b16d3551f980452f40f25454aed: Status 404 returned error can't find the container with id 09645d52b34e335da8c417d1715c28c0fbe03b16d3551f980452f40f25454aed Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.764617 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bdf996_67e4_41f3_95fb_798cc6c89502.slice/crio-ea27501603a4bc242697f78c42d5164616139995be6bb5510b66b4daf4405f4f WatchSource:0}: Error finding container ea27501603a4bc242697f78c42d5164616139995be6bb5510b66b4daf4405f4f: Status 404 returned error can't find the container with id ea27501603a4bc242697f78c42d5164616139995be6bb5510b66b4daf4405f4f Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.777355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.782281 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5v5jq" Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.785388 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fc1488_82a5_4f0a_a201_4d614587a448.slice/crio-f0d426c9bf333fe2b563deab9e88bc8aee7cd380003b396b0070cc2096064621 WatchSource:0}: Error finding container f0d426c9bf333fe2b563deab9e88bc8aee7cd380003b396b0070cc2096064621: Status 404 returned error can't find the container with id f0d426c9bf333fe2b563deab9e88bc8aee7cd380003b396b0070cc2096064621 Jul 18 11:48:52.803949 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:48:52.791695 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe06274_ed0d_4b45_b177_61cd75f4ad1f.slice/crio-8a2a171bfa0623dbe038fa568298e739878f52d6208713c1d6361c81b2fc1ab4 WatchSource:0}: Error finding container 8a2a171bfa0623dbe038fa568298e739878f52d6208713c1d6361c81b2fc1ab4: Status 404 returned error can't find the container with id 8a2a171bfa0623dbe038fa568298e739878f52d6208713c1d6361c81b2fc1ab4 Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.915875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:52.915907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916020 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916102 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:48:53.916086974 +0000 UTC m=+2.988146458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916025 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916139 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916149 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:52.962413 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:52.916189 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:53.916177994 +0000 UTC m=+2.988237474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:53.035888 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.016455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:53.035888 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.016645 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:53.035888 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.016706 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:54.016688922 +0000 UTC m=+3.088748406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:53.396055 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.336972 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 11:43:52 +0000 UTC" deadline="2028-01-20 19:17:29.730993083 +0000 UTC" Jul 18 11:48:53.396055 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.337028 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13231h28m36.393968687s" Jul 18 11:48:53.419631 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.419545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:53.420221 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.419657 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:48:53.420221 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.420128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:53.420317 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.420233 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:48:53.444350 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.444151 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:53.455353 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.455300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-49tjp" event={"ID":"65fca35e-cf89-44f8-ac27-6d26f9b1753a","Type":"ContainerStarted","Data":"72b69ca97e5d62a1d265b600e5875a0d6e08966a1233feef90f928e467584599"} Jul 18 11:48:53.458593 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.458348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7c5sk" event={"ID":"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c","Type":"ContainerStarted","Data":"6c89f27f15507d01e37601fb3a02d64e7eb67247d8786ce24f07b24b62234b25"} Jul 18 11:48:53.461663 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.461622 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"f5797e5b85abf8a89c911a3546660467","Type":"ContainerStarted","Data":"b61e4c66dfd54fa02bafdec64c4b92bded094c1d668dc0bd77acce976d5518c3"} Jul 18 11:48:53.464871 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.464811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5v5jq" event={"ID":"0fe06274-ed0d-4b45-b177-61cd75f4ad1f","Type":"ContainerStarted","Data":"8a2a171bfa0623dbe038fa568298e739878f52d6208713c1d6361c81b2fc1ab4"} Jul 18 11:48:53.477946 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.477916 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"ea27501603a4bc242697f78c42d5164616139995be6bb5510b66b4daf4405f4f"} Jul 18 11:48:53.489368 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.485736 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerStarted","Data":"09645d52b34e335da8c417d1715c28c0fbe03b16d3551f980452f40f25454aed"} Jul 18 11:48:53.495094 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.492742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jt2z9" event={"ID":"1e7deda4-422f-4caa-93f0-876b65fdb720","Type":"ContainerStarted","Data":"1d317cf60afb672d17108d69c24d79b2f677a5165b8fd57fb7fd173943ec4cd4"} Jul 18 11:48:53.500009 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.499909 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" event={"ID":"5dd3d632-53fe-4f73-b152-c10013ca4bc4","Type":"ContainerStarted","Data":"897acb817724d8d76da9fd9a289a55d6889bdd8229ba96e0bea2a40ed3226a79"} Jul 18 11:48:53.516032 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.516000 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wppmv" event={"ID":"cc65839f-dcf6-4f4e-8320-acb7d2dde8be","Type":"ContainerStarted","Data":"a996d02fdd49c964b7a853d10795ca6751e8c1b5dce4207ecb31566deb803aa6"} Jul 18 11:48:53.523169 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.523140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" event={"ID":"5dcb4fafed7c79e7e7353434bd1881be","Type":"ContainerStarted","Data":"b9db0803e72f53b262b36f3355bd029833cae46b640e8d3c9d7aec57da3331eb"} Jul 18 11:48:53.532217 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.532185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" event={"ID":"a5fc1488-82a5-4f0a-a201-4d614587a448","Type":"ContainerStarted","Data":"f0d426c9bf333fe2b563deab9e88bc8aee7cd380003b396b0070cc2096064621"} Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.923210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:53.923255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923388 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923403 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923424 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923437 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923470 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:48:55.923449893 +0000 UTC m=+4.995509386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:53.923558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:53.923500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:55.923480334 +0000 UTC m=+4.995539819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:54.024246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:54.024206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:54.024412 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:54.024358 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:54.024470 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:54.024430 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:56.02441228 +0000 UTC m=+5.096471764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:54.181845 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:54.181764 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:54.337590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:54.337550 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 11:43:52 +0000 UTC" deadline="2028-02-09 22:06:12.066729193 +0000 UTC" Jul 18 11:48:54.337590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:54.337586 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13714h17m17.729147184s" Jul 18 11:48:54.406705 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:54.406676 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:54.406881 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:54.406807 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:48:55.381023 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:55.380995 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 11:48:55.408781 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:55.408749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:55.408940 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.408909 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:48:55.409418 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:55.409394 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:55.409512 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.409493 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:48:55.943406 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:55.943374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:55.943591 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:55.943419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:55.943591 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.943571 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:48:55.943694 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.943596 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:48:55.943694 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.943608 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:55.943694 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.943659 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:48:59.943642406 +0000 UTC m=+9.015701905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:55.944059 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.944026 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:55.944144 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:55.944098 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:48:59.944083009 +0000 UTC m=+9.016142495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:56.045185 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:56.045124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:56.045358 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:56.045271 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:56.045358 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:56.045348 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:00.045328619 +0000 UTC m=+9.117388119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:48:56.406337 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:56.406302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:56.406844 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:56.406435 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:48:57.409320 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:57.408681 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:57.409320 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:57.408820 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:48:57.409320 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:57.408895 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:57.409320 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:57.408968 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:48:58.406470 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:58.406442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:48:58.406658 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:58.406574 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:48:59.406336 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:59.406304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:59.406749 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:59.406347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:59.406749 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.406455 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:48:59.406947 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.406918 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:48:59.977176 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:59.977146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:48:59.977390 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:48:59.977181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:48:59.977390 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977336 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:48:59.977390 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977356 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:48:59.977390 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977372 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:59.977529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977422 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:07.977409513 +0000 UTC m=+17.049468994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:48:59.977529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977475 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:48:59.977529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:48:59.977497 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:49:07.977489029 +0000 UTC m=+17.049548510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:49:00.077819 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:00.077770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:00.077989 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:00.077924 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:00.077989 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:00.077984 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:08.077970245 +0000 UTC m=+17.150029739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:00.406595 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:00.406485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:00.407013 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:00.406621 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:01.407922 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:01.407894 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:01.408363 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:01.408016 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:01.408363 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:01.408356 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:01.408475 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:01.408428 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:02.405779 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:02.405744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:02.405977 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:02.405887 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:03.406417 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:03.406382 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:03.406936 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:03.406386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:03.406936 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:03.406519 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:03.406936 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:03.406586 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:04.406367 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:04.406333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:04.406519 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:04.406461 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:05.406481 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:05.406451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:05.406652 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:05.406462 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:05.406652 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:05.406593 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:05.407017 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:05.406651 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:06.406693 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:06.406658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:06.407105 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:06.406802 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:07.406677 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:07.406641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:07.406860 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:07.406793 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:07.406860 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:07.406821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:07.407246 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:07.406931 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:08.036094 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:08.036036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:08.036107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036221 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036244 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036258 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036269 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036303 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:49:24.036278716 +0000 UTC m=+33.108338211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:49:08.036316 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.036322 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:24.036313583 +0000 UTC m=+33.108373068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:49:08.137454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:08.137417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:08.137596 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.137534 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:08.137596 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.137588 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:24.137575655 +0000 UTC m=+33.209635137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:08.406432 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:08.406353 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:08.406576 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:08.406463 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:09.406495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:09.406465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:09.406924 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:09.406465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:09.406924 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:09.406597 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:09.406924 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:09.406641 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:10.406513 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:10.406478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:10.406911 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:10.406620 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:11.406428 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:11.406400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:11.406606 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:11.406473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:11.406606 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:11.406546 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:11.406989 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:11.406642 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:12.406708 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:12.406672 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:12.407073 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:12.406775 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:13.406420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.406261 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:13.406560 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.406315 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:13.406560 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:13.406508 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:13.406666 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:13.406564 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:13.575678 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.575648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5v5jq" event={"ID":"0fe06274-ed0d-4b45-b177-61cd75f4ad1f","Type":"ContainerStarted","Data":"abbda2eb3dff810c33bc783c6ffebfc9b879966fb85e1e31843a2d3f6469631c"} Jul 18 11:49:13.577414 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.577394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"adc55bfd284e09686a6e0ce9a0fe46837d393a15add64886d53814de7a0e7227"} Jul 18 11:49:13.577497 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.577451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"020695fd382d8115c04046aeb5c452be8f5b852137dadb9cbae463e9c5dbea6f"} Jul 18 11:49:13.578840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.578819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" event={"ID":"5dd3d632-53fe-4f73-b152-c10013ca4bc4","Type":"ContainerStarted","Data":"4b6a8347afbb615a91e96b61061faf62c7f0ffa985a764a2192a7a339a1e9fe4"} Jul 18 11:49:13.580607 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.580586 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" event={"ID":"5dcb4fafed7c79e7e7353434bd1881be","Type":"ContainerStarted","Data":"93aa9c754e450fbdd1b4c8d5573f940e2982bc6ebcf69e99d74a428cca90cf55"} Jul 18 11:49:13.597095 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.597034 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5v5jq" podStartSLOduration=2.650289805 podStartE2EDuration="22.597021848s" totalImagesPullingTime="19.946732043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:13.596875794 +0000 UTC m=+22.668935297" watchObservedRunningTime="2026-07-18 11:49:13.597021848 +0000 UTC m=+22.669081350" Jul 18 11:49:13.609695 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.609658 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-242.ec2.internal" podStartSLOduration=21.609647164 podStartE2EDuration="21.609647164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:13.608845503 +0000 UTC m=+22.680905060" watchObservedRunningTime="2026-07-18 11:49:13.609647164 +0000 UTC m=+22.681706666" Jul 18 11:49:13.625417 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:13.625381 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2l7rc" podStartSLOduration=2.572627373 podStartE2EDuration="22.625368541s" totalImagesPullingTime="20.052741168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:13.624883591 +0000 UTC m=+22.696943101" watchObservedRunningTime="2026-07-18 11:49:13.625368541 +0000 UTC m=+22.697428044" Jul 18 11:49:14.406381 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.406108 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:14.406679 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:14.406440 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:14.584673 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.584640 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" event={"ID":"a5fc1488-82a5-4f0a-a201-4d614587a448","Type":"ContainerStarted","Data":"8f6f25aa55cdf14e59636db6567c7ba4fc8b6384ff0d1566cf34b228945a6332"} Jul 18 11:49:14.586068 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.586025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-49tjp" event={"ID":"65fca35e-cf89-44f8-ac27-6d26f9b1753a","Type":"ContainerStarted","Data":"ffe1a657eff4b3cfb49ffe420e9ab76df93a6389edd71063192a08d592869a55"} Jul 18 11:49:14.587656 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.587633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7c5sk" event={"ID":"a0b12eaf-f8e1-4431-9f83-c6ecb863b32c","Type":"ContainerStarted","Data":"9a7ffb30517017c96ee063e42916fb701306058e783ab8f492d86875891cc65f"} Jul 18 11:49:14.589178 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.589153 2578 generic.go:356] "Generic (PLEG): container finished" podID="f5797e5b85abf8a89c911a3546660467" containerID="cecd3c972cd81f2284683f9dfd887a0644d7ff0468dca377673421692779a6cb" exitCode=0 Jul 18 11:49:14.589284 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.589235 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"f5797e5b85abf8a89c911a3546660467","Type":"ContainerDied","Data":"cecd3c972cd81f2284683f9dfd887a0644d7ff0468dca377673421692779a6cb"} Jul 18 11:49:14.591911 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.591894 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:49:14.592212 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592195 2578 generic.go:356] "Generic (PLEG): container finished" podID="85bdf996-67e4-41f3-95fb-798cc6c89502" containerID="adc55bfd284e09686a6e0ce9a0fe46837d393a15add64886d53814de7a0e7227" exitCode=1 Jul 18 11:49:14.592281 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592250 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerDied","Data":"adc55bfd284e09686a6e0ce9a0fe46837d393a15add64886d53814de7a0e7227"} Jul 18 11:49:14.592335 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"1ea3090bb3aff9c4b994ba8f92cbc809561c5d582c6e51b59def13d4b8da432f"} Jul 18 11:49:14.592376 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592332 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"3bbbf3b3955e3adfc9c878d68ced215f242426d5dcbb7d56f5d8c28fbde6c3c7"} Jul 18 11:49:14.592376 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592373 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"62d8af86d9e2ef984c4e1fa94b202f1b31b7cf6342928957f8ee91ca945686c3"} Jul 18 11:49:14.592464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.592402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"14e2d8315fdd700630af15a9d27f65cb584961aed445a6cecbcf28eeaedd86d6"} Jul 18 11:49:14.594133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.593645 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="9767d84232e5f163ab0ed6e354c5ba2ca9c59c1362e7f54d8c045fb61232bbef" exitCode=0 Jul 18 11:49:14.594133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.593682 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"9767d84232e5f163ab0ed6e354c5ba2ca9c59c1362e7f54d8c045fb61232bbef"} Jul 18 11:49:14.595591 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.595567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jt2z9" event={"ID":"1e7deda4-422f-4caa-93f0-876b65fdb720","Type":"ContainerStarted","Data":"55e042beedaf432624dea27815cb231ddeb46e9c917bee10000d5619e12156d1"} Jul 18 11:49:14.597162 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.597140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wppmv" event={"ID":"cc65839f-dcf6-4f4e-8320-acb7d2dde8be","Type":"ContainerStarted","Data":"35cc9033ea7d303d32b8f496ee69adff1f40e00aaa56e58d120db7eb191f9f00"} Jul 18 11:49:14.600981 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.600943 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-49tjp" podStartSLOduration=3.631135825 podStartE2EDuration="23.600932976s" totalImagesPullingTime="19.969797151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:14.600784512 +0000 UTC m=+23.672844014" watchObservedRunningTime="2026-07-18 11:49:14.600932976 +0000 UTC m=+23.672992479" Jul 18 11:49:14.614798 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.614754 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jt2z9" podStartSLOduration=3.602950195 podStartE2EDuration="23.614742781s" totalImagesPullingTime="20.011792586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:14.614626323 +0000 UTC m=+23.686685826" watchObservedRunningTime="2026-07-18 11:49:14.614742781 +0000 UTC m=+23.686802284" Jul 18 11:49:14.680619 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.680570 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wppmv" podStartSLOduration=3.611004988 podStartE2EDuration="23.680558285s" totalImagesPullingTime="20.069553297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:14.667164616 +0000 UTC m=+23.739224120" watchObservedRunningTime="2026-07-18 11:49:14.680558285 +0000 UTC m=+23.752617788" Jul 18 11:49:14.680758 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:14.680737 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7c5sk" podStartSLOduration=3.629896692 podStartE2EDuration="23.68073034s" totalImagesPullingTime="20.050833648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:14.680652086 +0000 UTC m=+23.752711590" watchObservedRunningTime="2026-07-18 11:49:14.68073034 +0000 UTC m=+23.752789843" Jul 18 11:49:15.066007 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.065811 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 11:49:15.397564 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.397435 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T11:49:15.066004544Z","UUID":"f13fde42-e3ec-45b7-b9c8-432ea0457518","Handler":null,"Name":"","Endpoint":""} Jul 18 11:49:15.398725 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.398706 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 11:49:15.398837 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.398746 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 11:49:15.406266 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.406221 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:15.406357 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.406231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:15.406357 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:15.406346 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:15.406525 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:15.406376 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:15.601364 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.601314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" event={"ID":"a5fc1488-82a5-4f0a-a201-4d614587a448","Type":"ContainerStarted","Data":"726ee43d481b92d2df88e7a6cb3e4e9bd30670622cc6acd18c0b709842d45240"} Jul 18 11:49:15.603272 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.603233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" event={"ID":"f5797e5b85abf8a89c911a3546660467","Type":"ContainerStarted","Data":"e19a406fc14af42379870fe96a310480cd342c095d44c45da155125c650500d0"} Jul 18 11:49:15.618027 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:15.617986 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-242.ec2.internal" podStartSLOduration=23.617972341 podStartE2EDuration="23.617972341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:15.617154166 +0000 UTC m=+24.689213663" watchObservedRunningTime="2026-07-18 11:49:15.617972341 +0000 UTC m=+24.690031845" Jul 18 11:49:16.369084 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.369005 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:49:16.406698 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.406672 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:16.406854 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:16.406777 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:16.608872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.608846 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:49:16.609380 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.609323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"ea472799f03d68b16fb1dac224bab2a1b3736e8ae985aef272b668d73f041f64"} Jul 18 11:49:16.611571 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.611544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" event={"ID":"a5fc1488-82a5-4f0a-a201-4d614587a448","Type":"ContainerStarted","Data":"9666c67afdac4a8149137904b3dd6689eee847ab4695972ff2d1b809f77f1505"} Jul 18 11:49:16.632210 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.632128 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sq2mq" podStartSLOduration=3.469562467 podStartE2EDuration="25.632113548s" totalImagesPullingTime="22.162551081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:16.631550745 +0000 UTC m=+25.703610261" watchObservedRunningTime="2026-07-18 11:49:16.632113548 +0000 UTC m=+25.704173051" Jul 18 11:49:16.748181 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.748136 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:49:16.748909 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:16.748891 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:49:17.405883 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:17.405852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:17.406061 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:17.405852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:17.406061 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:17.405965 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:17.406061 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:17.406027 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:17.614512 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:17.614485 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7c5sk" Jul 18 11:49:18.405912 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:18.405881 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:18.406078 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:18.405983 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:19.406787 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.406624 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:19.407372 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.406670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:19.407372 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:19.406873 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:19.407372 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:19.406921 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:19.621761 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.621695 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:49:19.622105 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.622074 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"00eb33fb3dc1349766ceb40144b9f91686946852853764d066ce29bee2549d2e"} Jul 18 11:49:19.622384 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.622301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:19.622481 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.622446 2578 scope.go:117] "RemoveContainer" containerID="adc55bfd284e09686a6e0ce9a0fe46837d393a15add64886d53814de7a0e7227" Jul 18 11:49:19.622546 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.622492 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:19.622602 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.622565 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:19.624323 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.624299 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="55354ea9c4e55d2b988850168600ee4bba8cba53a7ad198b3ce03cbd2c4b6718" exitCode=0 Jul 18 11:49:19.624416 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.624399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"55354ea9c4e55d2b988850168600ee4bba8cba53a7ad198b3ce03cbd2c4b6718"} Jul 18 11:49:19.639212 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.639193 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:19.639924 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:19.639911 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:20.405943 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.405871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:20.406125 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:20.405986 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:20.631038 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.631005 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:49:20.631562 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.631367 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" event={"ID":"85bdf996-67e4-41f3-95fb-798cc6c89502","Type":"ContainerStarted","Data":"5768f8f271ab7a5d82e16ea8e94907b2d5c8fb87b7e8253bda46e7dcf7702f0d"} Jul 18 11:49:20.658135 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.658033 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" podStartSLOduration=9.123966169 podStartE2EDuration="29.658020298s" totalImagesPullingTime="20.534054129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:20.656413734 +0000 UTC m=+29.728473238" watchObservedRunningTime="2026-07-18 11:49:20.658020298 +0000 UTC m=+29.730079779" Jul 18 11:49:20.939305 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.939144 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-998xh"] Jul 18 11:49:20.939429 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.939379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:20.939469 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:20.939450 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:20.941794 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.941770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ndsm5"] Jul 18 11:49:20.941952 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.941861 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:20.942010 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:20.941946 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:20.942530 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.942508 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wh7n8"] Jul 18 11:49:20.942636 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:20.942609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:20.942722 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:20.942703 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:21.634888 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:21.634853 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="a3d9703994758c19ee9c94d95c653d915a284c11d211f226ad749604a2e2a33f" exitCode=0 Jul 18 11:49:21.635264 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:21.634944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"a3d9703994758c19ee9c94d95c653d915a284c11d211f226ad749604a2e2a33f"} Jul 18 11:49:22.406537 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:22.406259 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:22.406537 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:22.406391 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:22.406537 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:22.406428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:22.406787 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:22.406550 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:22.640133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:22.640032 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="e70f208249ae2bd5b4f7a06988fcc9bb561869c7a11e02a32a35b5c9399065a4" exitCode=0 Jul 18 11:49:22.640133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:22.640088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"e70f208249ae2bd5b4f7a06988fcc9bb561869c7a11e02a32a35b5c9399065a4"} Jul 18 11:49:23.406136 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:23.406060 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:23.406323 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:23.406187 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:24.058997 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:24.058959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:24.059002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059141 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059163 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059186 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059199 2578 projected.go:196] Error preparing data for projected volume kube-api-access-vdbzw for pod openshift-network-diagnostics/network-check-target-998xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059215 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:49:56.059194832 +0000 UTC m=+65.131254317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 11:49:24.059393 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.059239 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw podName:f104962d-a24a-4034-a032-6dcf02384419 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:56.059227662 +0000 UTC m=+65.131287148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdbzw" (UniqueName: "kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw") pod "network-check-target-998xh" (UID: "f104962d-a24a-4034-a032-6dcf02384419") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 11:49:24.159522 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:24.159487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:24.159665 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.159589 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:24.159665 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.159638 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret podName:c73af55a-7e9a-416b-8144-38477c907ba0 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:56.159626232 +0000 UTC m=+65.231685714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret") pod "global-pull-secret-syncer-ndsm5" (UID: "c73af55a-7e9a-416b-8144-38477c907ba0") : object "kube-system"/"original-pull-secret" not registered Jul 18 11:49:24.405923 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:24.405846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:24.406108 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:24.405852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:24.406108 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.405957 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:24.406214 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:24.406104 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:25.406432 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:25.406393 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:25.407016 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:25.406501 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ndsm5" podUID="c73af55a-7e9a-416b-8144-38477c907ba0" Jul 18 11:49:26.406066 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.406009 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:26.406242 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.406162 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-998xh" podUID="f104962d-a24a-4034-a032-6dcf02384419" Jul 18 11:49:26.406242 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.406213 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:26.406374 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.406349 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wh7n8" podUID="e785910e-4713-4292-acc8-d9d884f9d8db" Jul 18 11:49:26.726820 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.726783 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-242.ec2.internal" event="NodeReady" Jul 18 11:49:26.727293 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.726924 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 11:49:26.758118 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.758082 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:49:26.763017 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.762996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.765136 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.765112 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 11:49:26.765267 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.765138 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 11:49:26.765267 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.765221 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 11:49:26.765476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.765460 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4x2w\"" Jul 18 11:49:26.770556 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.770533 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 11:49:26.772960 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.772936 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:49:26.777139 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.777085 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7fwbh"] Jul 18 11:49:26.780855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.780837 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mm94p"] Jul 18 11:49:26.780983 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.780967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.783288 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.783260 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 11:49:26.783288 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.783268 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 11:49:26.783288 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.783283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 11:49:26.783509 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.783297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fbzht\"" Jul 18 11:49:26.784501 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.784485 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.786554 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.786537 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qxztp\"" Jul 18 11:49:26.786796 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.786779 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 11:49:26.787104 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.787031 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 11:49:26.789155 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.788995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fwbh"] Jul 18 11:49:26.789818 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.789462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mm94p"] Jul 18 11:49:26.879998 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.879969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.879998 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9qtx\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e156db75-f982-4a5e-8cbe-8b06b24aa23f-config-volume\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jtl4\" (UniqueName: \"kubernetes.io/projected/cf18a13e-792a-4e89-a282-1d90683d414c-kube-api-access-6jtl4\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880225 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880195 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.880521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.880521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqkl7\" (UniqueName: \"kubernetes.io/projected/e156db75-f982-4a5e-8cbe-8b06b24aa23f-kube-api-access-zqkl7\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.880521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e156db75-f982-4a5e-8cbe-8b06b24aa23f-tmp-dir\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.880521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.880388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf18a13e-792a-4e89-a282-1d90683d414c-cert\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.937649 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.937617 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:26.981646 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.981646 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9qtx\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.981646 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e156db75-f982-4a5e-8cbe-8b06b24aa23f-config-volume\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jtl4\" (UniqueName: \"kubernetes.io/projected/cf18a13e-792a-4e89-a282-1d90683d414c-kube-api-access-6jtl4\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.981739 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.981759 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.981841 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:27.481823578 +0000 UTC m=+36.553883063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:26.981876 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981874 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.981978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqkl7\" (UniqueName: \"kubernetes.io/projected/e156db75-f982-4a5e-8cbe-8b06b24aa23f-kube-api-access-zqkl7\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.982069 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e156db75-f982-4a5e-8cbe-8b06b24aa23f-tmp-dir\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:26.982119 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:27.482102442 +0000 UTC m=+36.554161923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf18a13e-792a-4e89-a282-1d90683d414c-cert\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.982356 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e156db75-f982-4a5e-8cbe-8b06b24aa23f-config-volume\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.982807 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982447 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e156db75-f982-4a5e-8cbe-8b06b24aa23f-tmp-dir\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.982807 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982659 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.982895 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.982809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.983592 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.983567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.986066 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.986025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.986158 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.986025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.986158 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.986133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf18a13e-792a-4e89-a282-1d90683d414c-cert\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.990443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.990418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jtl4\" (UniqueName: \"kubernetes.io/projected/cf18a13e-792a-4e89-a282-1d90683d414c-kube-api-access-6jtl4\") pod \"ingress-canary-7fwbh\" (UID: \"cf18a13e-792a-4e89-a282-1d90683d414c\") " pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:26.991392 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.991366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:26.994196 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.994177 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqkl7\" (UniqueName: \"kubernetes.io/projected/e156db75-f982-4a5e-8cbe-8b06b24aa23f-kube-api-access-zqkl7\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:26.994644 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:26.994625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9qtx\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:27.047742 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.047594 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:27.092998 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.092975 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7fwbh" Jul 18 11:49:27.257521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.257442 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7fwbh"] Jul 18 11:49:27.261633 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:27.261608 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf18a13e_792a_4e89_a282_1d90683d414c.slice/crio-1c9ae0fab4cb009e707313f69f994862ab924ed90c5c41ce991fa605bcea351f WatchSource:0}: Error finding container 1c9ae0fab4cb009e707313f69f994862ab924ed90c5c41ce991fa605bcea351f: Status 404 returned error can't find the container with id 1c9ae0fab4cb009e707313f69f994862ab924ed90c5c41ce991fa605bcea351f Jul 18 11:49:27.406030 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.405999 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:27.408578 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.408552 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 11:49:27.488330 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.488303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:27.488463 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.488353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:27.488463 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:27.488444 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:27.488529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:27.488460 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:27.488529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:27.488481 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:27.488529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:27.488491 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:28.488478065 +0000 UTC m=+37.560537546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:27.488529 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:27.488521 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:28.488510571 +0000 UTC m=+37.560570052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:27.653366 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.653292 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fwbh" event={"ID":"cf18a13e-792a-4e89-a282-1d90683d414c","Type":"ContainerStarted","Data":"1c9ae0fab4cb009e707313f69f994862ab924ed90c5c41ce991fa605bcea351f"} Jul 18 11:49:27.655702 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.655678 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="c9c33e25f379c4776af6d58c116540e8dcdd3a30d11d9770ac23c64e4c5ef055" exitCode=0 Jul 18 11:49:27.655810 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:27.655712 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"c9c33e25f379c4776af6d58c116540e8dcdd3a30d11d9770ac23c64e4c5ef055"} Jul 18 11:49:28.406595 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.406513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:28.407360 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.406514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:28.412905 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.412879 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 11:49:28.413755 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.413732 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 11:49:28.413755 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.413745 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6q4p9\"" Jul 18 11:49:28.413755 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.413750 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 11:49:28.413991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.413750 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qw87\"" Jul 18 11:49:28.496797 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.496767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:28.496956 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.496844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:28.496956 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:28.496936 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:28.497067 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:28.496944 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:28.497067 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:28.497037 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:28.497172 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:28.497023 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:30.497001839 +0000 UTC m=+39.569061323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:28.497172 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:28.497112 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:30.497096908 +0000 UTC m=+39.569156392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:28.662089 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.661993 2578 generic.go:356] "Generic (PLEG): container finished" podID="03b66d23-e64b-4076-9494-c10d0b7a891d" containerID="f8ff5be59350a265d2ed18406dc1b199a126d28dbd07e5ef8aae41f0be0b32e0" exitCode=0 Jul 18 11:49:28.662089 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:28.662062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerDied","Data":"f8ff5be59350a265d2ed18406dc1b199a126d28dbd07e5ef8aae41f0be0b32e0"} Jul 18 11:49:29.395140 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:29.394889 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:29.669069 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:29.669026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-srfjb" event={"ID":"03b66d23-e64b-4076-9494-c10d0b7a891d","Type":"ContainerStarted","Data":"776eeb5ac3557ab0a934f144e1b082e936583ba580cc022a1e16145e0bdb194b"} Jul 18 11:49:29.692865 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:29.692808 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-srfjb" podStartSLOduration=8.072536055 podStartE2EDuration="38.692790746s" totalImagesPullingTime="30.620254691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:29.692110058 +0000 UTC m=+38.764169569" watchObservedRunningTime="2026-07-18 11:49:29.692790746 +0000 UTC m=+38.764850228" Jul 18 11:49:29.814499 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:29.814326 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:29.913812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:29.913770 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" limit=1024 Jul 18 11:49:30.510036 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:30.510003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:30.510183 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:30.510106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:30.510183 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:30.510156 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:30.510314 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:30.510223 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:34.510208439 +0000 UTC m=+43.582267920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:30.510314 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:30.510243 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:30.510314 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:30.510255 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:30.510314 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:30.510308 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:34.510295389 +0000 UTC m=+43.582354870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:30.672971 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:30.672932 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7fwbh" event={"ID":"cf18a13e-792a-4e89-a282-1d90683d414c","Type":"ContainerStarted","Data":"ac84306ff813d9577bb5d4daceb9970399d18598d2f720f7ccf2d4a163b5bca4"} Jul 18 11:49:30.687162 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:30.687109 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7fwbh" podStartSLOduration=1.5449577460000001 podStartE2EDuration="4.687092229s" totalImagesPullingTime="3.142134483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:30.686984202 +0000 UTC m=+39.759043719" watchObservedRunningTime="2026-07-18 11:49:30.687092229 +0000 UTC m=+39.759151733" Jul 18 11:49:34.539133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:34.539087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:34.539158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:34.539270 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:34.539273 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:34.539292 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:34.539323 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:42.539308832 +0000 UTC m=+51.611368336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:34.539586 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:34.539337 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:42.539330888 +0000 UTC m=+51.611390369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:41.670674 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.670643 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb"] Jul 18 11:49:41.681731 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.681710 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx"] Jul 18 11:49:41.681857 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.681839 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.684223 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.684198 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 18 11:49:41.684347 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.684301 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 18 11:49:41.684673 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.684656 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dnb2b\"" Jul 18 11:49:41.696647 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.696631 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v"] Jul 18 11:49:41.696783 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.696761 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.698938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.698920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 11:49:41.700042 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.700018 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 11:49:41.700149 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.700067 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 18 11:49:41.701001 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.700989 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cjg2l\"" Jul 18 11:49:41.701500 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.701480 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 18 11:49:41.711300 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.711281 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb"] Jul 18 11:49:41.711379 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.711308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v"] Jul 18 11:49:41.711379 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.711325 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx"] Jul 18 11:49:41.711448 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.711387 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.713709 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.713688 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:49:41.713709 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.713705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 18 11:49:41.713852 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.713767 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 18 11:49:41.713852 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.713806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cl9kb\"" Jul 18 11:49:41.777906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.777880 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc"] Jul 18 11:49:41.792599 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.792579 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6"] Jul 18 11:49:41.792738 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.792721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" Jul 18 11:49:41.793119 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfthg\" (UniqueName: \"kubernetes.io/projected/6360c550-5517-46fc-adbb-5730bf3a31b8-kube-api-access-qfthg\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.793219 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.793219 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.793345 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793224 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crjcg\" (UniqueName: \"kubernetes.io/projected/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-kube-api-access-crjcg\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.793345 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.793445 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d2213a6-b613-4961-9d5d-6b483a05b011-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.793445 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.793425 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.797297 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.797230 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k62lt\"" Jul 18 11:49:41.801802 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.801774 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc"] Jul 18 11:49:41.801802 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.801800 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6"] Jul 18 11:49:41.801917 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.801862 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.803967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.803949 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 18 11:49:41.804064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.803971 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-n87dd\"" Jul 18 11:49:41.804064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.803985 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 18 11:49:41.804064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.803971 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:49:41.804296 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.804283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 18 11:49:41.894446 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snkq8\" (UniqueName: \"kubernetes.io/projected/52094835-811b-4c18-8436-f8e9d0aaf34e-kube-api-access-snkq8\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.894601 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.894601 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52094835-811b-4c18-8436-f8e9d0aaf34e-config\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.894694 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n249j\" (UniqueName: \"kubernetes.io/projected/d1ddff66-f9e8-4295-b40b-133dd9d30fba-kube-api-access-n249j\") pod \"network-check-source-7bc5b6bf85-zqdxc\" (UID: \"d1ddff66-f9e8-4295-b40b-133dd9d30fba\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" Jul 18 11:49:41.894694 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894620 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 11:49:41.894694 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.894694 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894676 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls podName:6360c550-5517-46fc-adbb-5730bf3a31b8 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:42.394657548 +0000 UTC m=+51.466717034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6r88v" (UID: "6360c550-5517-46fc-adbb-5730bf3a31b8") : secret "samples-operator-tls" not found Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crjcg\" (UniqueName: \"kubernetes.io/projected/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-kube-api-access-crjcg\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894717 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52094835-811b-4c18-8436-f8e9d0aaf34e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894754 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:42.39474311 +0000 UTC m=+51.466802604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d2213a6-b613-4961-9d5d-6b483a05b011-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.894882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.895170 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894910 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:41.895170 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.894911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfthg\" (UniqueName: \"kubernetes.io/projected/6360c550-5517-46fc-adbb-5730bf3a31b8-kube-api-access-qfthg\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.895170 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:41.894998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:42.394955119 +0000 UTC m=+51.467014604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:41.895557 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.895540 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.895607 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.895585 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5d2213a6-b613-4961-9d5d-6b483a05b011-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:41.909429 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.909401 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crjcg\" (UniqueName: \"kubernetes.io/projected/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-kube-api-access-crjcg\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:41.909429 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.909417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfthg\" (UniqueName: \"kubernetes.io/projected/6360c550-5517-46fc-adbb-5730bf3a31b8-kube-api-access-qfthg\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:41.996077 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.995983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snkq8\" (UniqueName: \"kubernetes.io/projected/52094835-811b-4c18-8436-f8e9d0aaf34e-kube-api-access-snkq8\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.996077 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.996020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52094835-811b-4c18-8436-f8e9d0aaf34e-config\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.996077 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.996036 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n249j\" (UniqueName: \"kubernetes.io/projected/d1ddff66-f9e8-4295-b40b-133dd9d30fba-kube-api-access-n249j\") pod \"network-check-source-7bc5b6bf85-zqdxc\" (UID: \"d1ddff66-f9e8-4295-b40b-133dd9d30fba\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" Jul 18 11:49:41.996284 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.996179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52094835-811b-4c18-8436-f8e9d0aaf34e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.996624 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.996607 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52094835-811b-4c18-8436-f8e9d0aaf34e-config\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:41.998252 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:41.998235 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52094835-811b-4c18-8436-f8e9d0aaf34e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:42.003322 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.003297 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snkq8\" (UniqueName: \"kubernetes.io/projected/52094835-811b-4c18-8436-f8e9d0aaf34e-kube-api-access-snkq8\") pod \"service-ca-operator-5bcbc94c78-ndvr6\" (UID: \"52094835-811b-4c18-8436-f8e9d0aaf34e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:42.003700 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.003681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n249j\" (UniqueName: \"kubernetes.io/projected/d1ddff66-f9e8-4295-b40b-133dd9d30fba-kube-api-access-n249j\") pod \"network-check-source-7bc5b6bf85-zqdxc\" (UID: \"d1ddff66-f9e8-4295-b40b-133dd9d30fba\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" Jul 18 11:49:42.102762 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.102726 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" Jul 18 11:49:42.109889 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.109866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" Jul 18 11:49:42.249153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.249020 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc"] Jul 18 11:49:42.252476 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:42.252452 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1ddff66_f9e8_4295_b40b_133dd9d30fba.slice/crio-1d712c911dd86b92dd3f579462fc84faa5d23bbe067d60d5b47d123327effc14 WatchSource:0}: Error finding container 1d712c911dd86b92dd3f579462fc84faa5d23bbe067d60d5b47d123327effc14: Status 404 returned error can't find the container with id 1d712c911dd86b92dd3f579462fc84faa5d23bbe067d60d5b47d123327effc14 Jul 18 11:49:42.260618 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.260600 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6"] Jul 18 11:49:42.263290 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:42.263269 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52094835_811b_4c18_8436_f8e9d0aaf34e.slice/crio-d43d24b3bc50235ef347c6e5ca1c579847ec793989d73a829fc1b41cc9124e2a WatchSource:0}: Error finding container d43d24b3bc50235ef347c6e5ca1c579847ec793989d73a829fc1b41cc9124e2a: Status 404 returned error can't find the container with id d43d24b3bc50235ef347c6e5ca1c579847ec793989d73a829fc1b41cc9124e2a Jul 18 11:49:42.399307 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.399278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:42.399459 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.399360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:42.399459 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399424 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:42.399459 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399440 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 11:49:42.399558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399480 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:43.399467069 +0000 UTC m=+52.471526555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:42.399558 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399495 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls podName:6360c550-5517-46fc-adbb-5730bf3a31b8 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:43.399488284 +0000 UTC m=+52.471547765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6r88v" (UID: "6360c550-5517-46fc-adbb-5730bf3a31b8") : secret "samples-operator-tls" not found Jul 18 11:49:42.399558 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.399512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:42.399657 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399583 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:42.399657 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.399616 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:43.399605461 +0000 UTC m=+52.471664947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:42.601377 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.601290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:42.601377 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.601370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:42.601555 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.601436 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 11:49:42.601555 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.601478 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 11:49:42.601555 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.601488 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd96cb6c-kpns4: secret "image-registry-tls" not found Jul 18 11:49:42.601555 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.601503 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls podName:e156db75-f982-4a5e-8cbe-8b06b24aa23f nodeName:}" failed. No retries permitted until 2026-07-18 11:49:58.601489014 +0000 UTC m=+67.673548498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls") pod "dns-default-mm94p" (UID: "e156db75-f982-4a5e-8cbe-8b06b24aa23f") : secret "dns-default-metrics-tls" not found Jul 18 11:49:42.601555 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:42.601523 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls podName:c2aa38c0-d758-49c1-ad20-3a614100e76c nodeName:}" failed. No retries permitted until 2026-07-18 11:49:58.601513677 +0000 UTC m=+67.673573158 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls") pod "image-registry-cd96cb6c-kpns4" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c") : secret "image-registry-tls" not found Jul 18 11:49:42.700903 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.700867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" event={"ID":"52094835-811b-4c18-8436-f8e9d0aaf34e","Type":"ContainerStarted","Data":"d43d24b3bc50235ef347c6e5ca1c579847ec793989d73a829fc1b41cc9124e2a"} Jul 18 11:49:42.702588 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:42.702562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" event={"ID":"d1ddff66-f9e8-4295-b40b-133dd9d30fba","Type":"ContainerStarted","Data":"1d712c911dd86b92dd3f579462fc84faa5d23bbe067d60d5b47d123327effc14"} Jul 18 11:49:43.408550 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:43.408510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:43.408550 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:43.408556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:43.408766 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:43.408610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:43.408766 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408675 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 11:49:43.408766 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408733 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:43.408766 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls podName:6360c550-5517-46fc-adbb-5730bf3a31b8 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:45.408729545 +0000 UTC m=+54.480789031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6r88v" (UID: "6360c550-5517-46fc-adbb-5730bf3a31b8") : secret "samples-operator-tls" not found Jul 18 11:49:43.408937 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408771 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:45.408760889 +0000 UTC m=+54.480820370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:43.408937 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408809 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:43.408937 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:43.408837 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:45.408826258 +0000 UTC m=+54.480885740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:45.427374 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.427340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.427389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.427455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427494 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427535 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427569 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls podName:6360c550-5517-46fc-adbb-5730bf3a31b8 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:49.427549306 +0000 UTC m=+58.499608800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6r88v" (UID: "6360c550-5517-46fc-adbb-5730bf3a31b8") : secret "samples-operator-tls" not found Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427586 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:49.427576361 +0000 UTC m=+58.499635855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427623 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:45.427806 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:45.427680 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:49.427665811 +0000 UTC m=+58.499725302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:45.712465 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.712364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" event={"ID":"d1ddff66-f9e8-4295-b40b-133dd9d30fba","Type":"ContainerStarted","Data":"39c93bb125794b5a7b8cf9a2f97d910a0998d42d40740673400565f6158ab2ca"} Jul 18 11:49:45.713744 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.713716 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" event={"ID":"52094835-811b-4c18-8436-f8e9d0aaf34e","Type":"ContainerStarted","Data":"caf7c137a3a108b74d225943755e9e6ccc13e4520a015514538233c2807a409e"} Jul 18 11:49:45.727730 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.727687 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-zqdxc" podStartSLOduration=1.481643682 podStartE2EDuration="4.727675647s" totalImagesPullingTime="3.246031965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:45.726296491 +0000 UTC m=+54.798355994" watchObservedRunningTime="2026-07-18 11:49:45.727675647 +0000 UTC m=+54.799735149" Jul 18 11:49:45.740943 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:45.740905 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" podStartSLOduration=1.511208162 podStartE2EDuration="4.740892511s" totalImagesPullingTime="3.229684349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:45.740159455 +0000 UTC m=+54.812218971" watchObservedRunningTime="2026-07-18 11:49:45.740892511 +0000 UTC m=+54.812952018" Jul 18 11:49:46.777077 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.777028 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-psvl5"] Jul 18 11:49:46.779106 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.779090 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.781584 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.781563 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 11:49:46.781878 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.781863 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-29g54\"" Jul 18 11:49:46.782443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.782429 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 11:49:46.782523 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.782442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 11:49:46.782523 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.782442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 11:49:46.800805 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.800776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-psvl5"] Jul 18 11:49:46.838839 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.838806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-crio-socket\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.838984 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.838843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.838984 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.838914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-data-volume\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.839119 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.839014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.839119 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.839038 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5fl4\" (UniqueName: \"kubernetes.io/projected/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-api-access-k5fl4\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939491 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-crio-socket\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939648 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939648 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:46.939600 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:49:46.939648 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939596 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-crio-socket\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939648 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-data-volume\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939648 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:46.939645 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:47.439632688 +0000 UTC m=+56.511692169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:49:46.939949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5fl4\" (UniqueName: \"kubernetes.io/projected/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-api-access-k5fl4\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.939949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.939900 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-data-volume\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:46.940111 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:46.939948 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 11:49:46.940111 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:46.940002 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:47.439990188 +0000 UTC m=+56.512049686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : configmap "kube-rbac-proxy" not found Jul 18 11:49:46.947348 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:46.947321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5fl4\" (UniqueName: \"kubernetes.io/projected/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-api-access-k5fl4\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:47.444498 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:47.444467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:47.444780 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:47.444621 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:49:47.444780 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:47.444684 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:48.44467085 +0000 UTC m=+57.516730336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:49:47.444780 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:47.444675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:47.444780 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:47.444736 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 11:49:47.444945 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:47.444795 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:48.444780751 +0000 UTC m=+57.516840235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : configmap "kube-rbac-proxy" not found Jul 18 11:49:48.453271 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:48.453230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:48.453678 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:48.453362 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 11:49:48.453678 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:48.453399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:48.453678 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:48.453425 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:50.453410359 +0000 UTC m=+59.525469840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : configmap "kube-rbac-proxy" not found Jul 18 11:49:48.453678 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:48.453477 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:49:48.453678 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:48.453520 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:50.453509341 +0000 UTC m=+59.525568824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:49:48.870843 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:48.870769 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jt2z9_1e7deda4-422f-4caa-93f0-876b65fdb720/dns-node-resolver/0.log" Jul 18 11:49:49.460341 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:49.460300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:49.460341 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:49.460343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:49.460426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460445 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460501 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:57.460486813 +0000 UTC m=+66.532546297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460505 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460554 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:57.460542437 +0000 UTC m=+66.532601918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460566 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 11:49:49.460801 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:49.460608 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls podName:6360c550-5517-46fc-adbb-5730bf3a31b8 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:57.460595109 +0000 UTC m=+66.532654590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6r88v" (UID: "6360c550-5517-46fc-adbb-5730bf3a31b8") : secret "samples-operator-tls" not found Jul 18 11:49:49.670636 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:49.670611 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-49tjp_65fca35e-cf89-44f8-ac27-6d26f9b1753a/node-ca/0.log" Jul 18 11:49:50.468811 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:50.468775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:50.469206 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:50.468901 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 11:49:50.469206 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:50.468920 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:50.469206 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:50.468975 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:54.468959028 +0000 UTC m=+63.541018511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : configmap "kube-rbac-proxy" not found Jul 18 11:49:50.469206 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:50.469010 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:49:50.469206 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:50.469079 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:49:54.469065329 +0000 UTC m=+63.541124812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:49:50.674168 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:50.674137 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7fwbh_cf18a13e-792a-4e89-a282-1d90683d414c/serve-healthcheck-canary/0.log" Jul 18 11:49:51.645096 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:51.645064 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tqdw9" Jul 18 11:49:54.501870 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:54.501820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:54.502457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:54.501924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:49:54.502457 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:54.501961 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 11:49:54.502457 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:54.502068 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:02.502024031 +0000 UTC m=+71.574083512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : configmap "kube-rbac-proxy" not found Jul 18 11:49:54.502457 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:54.502089 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:49:54.502457 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:54.502147 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:02.502133113 +0000 UTC m=+71.574192615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:49:56.114064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.114010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:49:56.114064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.114066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:56.116510 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.116491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdbzw\" (UniqueName: \"kubernetes.io/projected/f104962d-a24a-4034-a032-6dcf02384419-kube-api-access-vdbzw\") pod \"network-check-target-998xh\" (UID: \"f104962d-a24a-4034-a032-6dcf02384419\") " pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:56.117026 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.117005 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 11:49:56.125010 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:56.124990 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 18 11:49:56.125118 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:56.125058 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs podName:e785910e-4713-4292-acc8-d9d884f9d8db nodeName:}" failed. No retries permitted until 2026-07-18 11:51:00.125031351 +0000 UTC m=+129.197090832 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs") pod "network-metrics-daemon-wh7n8" (UID: "e785910e-4713-4292-acc8-d9d884f9d8db") : secret "metrics-daemon-secret" not found Jul 18 11:49:56.214749 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.214720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:56.217087 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.217071 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 11:49:56.226977 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.226958 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c73af55a-7e9a-416b-8144-38477c907ba0-original-pull-secret\") pod \"global-pull-secret-syncer-ndsm5\" (UID: \"c73af55a-7e9a-416b-8144-38477c907ba0\") " pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:56.326367 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.326339 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qw87\"" Jul 18 11:49:56.334639 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.334617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:56.454242 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.454216 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-998xh"] Jul 18 11:49:56.457060 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:56.457019 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf104962d_a24a_4034_a032_6dcf02384419.slice/crio-3c70b0fcd4252513d002efdcea5a4ab43474a12541800c14634a1026412bcb1d WatchSource:0}: Error finding container 3c70b0fcd4252513d002efdcea5a4ab43474a12541800c14634a1026412bcb1d: Status 404 returned error can't find the container with id 3c70b0fcd4252513d002efdcea5a4ab43474a12541800c14634a1026412bcb1d Jul 18 11:49:56.514932 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.514906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ndsm5" Jul 18 11:49:56.628544 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.628513 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ndsm5"] Jul 18 11:49:56.631114 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:56.631067 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73af55a_7e9a_416b_8144_38477c907ba0.slice/crio-c5afec3ba3e3a65f08fee7f7b6e7c00498ab77244d0b5f63e6f7b8a907271fec WatchSource:0}: Error finding container c5afec3ba3e3a65f08fee7f7b6e7c00498ab77244d0b5f63e6f7b8a907271fec: Status 404 returned error can't find the container with id c5afec3ba3e3a65f08fee7f7b6e7c00498ab77244d0b5f63e6f7b8a907271fec Jul 18 11:49:56.742953 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.742925 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ndsm5" event={"ID":"c73af55a-7e9a-416b-8144-38477c907ba0","Type":"ContainerStarted","Data":"c5afec3ba3e3a65f08fee7f7b6e7c00498ab77244d0b5f63e6f7b8a907271fec"} Jul 18 11:49:56.744437 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.744419 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-998xh" event={"ID":"f104962d-a24a-4034-a032-6dcf02384419","Type":"ContainerStarted","Data":"9a09bb5bdd46808bf4a25f72dbfc77009915882ad195791bad7d78fa801bd98e"} Jul 18 11:49:56.744544 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.744452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-998xh" event={"ID":"f104962d-a24a-4034-a032-6dcf02384419","Type":"ContainerStarted","Data":"3c70b0fcd4252513d002efdcea5a4ab43474a12541800c14634a1026412bcb1d"} Jul 18 11:49:56.744605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.744541 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:49:56.759598 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:56.759558 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-998xh" podStartSLOduration=65.759546897 podStartE2EDuration="1m5.759546897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:56.757890724 +0000 UTC m=+65.829950252" watchObservedRunningTime="2026-07-18 11:49:56.759546897 +0000 UTC m=+65.831606377" Jul 18 11:49:57.526017 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.525978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:49:57.526017 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.526017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:49:57.526443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.526083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:57.526443 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:57.526164 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 11:49:57.526443 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:57.526186 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:57.526443 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:57.526234 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls podName:3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:13.526215684 +0000 UTC m=+82.598275187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2fmbx" (UID: "3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9") : secret "cluster-monitoring-operator-tls" not found Jul 18 11:49:57.526443 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:49:57.526249 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert podName:5d2213a6-b613-4961-9d5d-6b483a05b011 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:13.526241514 +0000 UTC m=+82.598300995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-dc5cb" (UID: "5d2213a6-b613-4961-9d5d-6b483a05b011") : secret "networking-console-plugin-cert" not found Jul 18 11:49:57.528678 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.528655 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6360c550-5517-46fc-adbb-5730bf3a31b8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6r88v\" (UID: \"6360c550-5517-46fc-adbb-5730bf3a31b8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:57.621616 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.621586 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cl9kb\"" Jul 18 11:49:57.629514 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.629488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" Jul 18 11:49:57.782591 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:57.782518 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v"] Jul 18 11:49:58.635236 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.635203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:58.635688 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.635342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:58.638254 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.638226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e156db75-f982-4a5e-8cbe-8b06b24aa23f-metrics-tls\") pod \"dns-default-mm94p\" (UID: \"e156db75-f982-4a5e-8cbe-8b06b24aa23f\") " pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:58.638563 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.638537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"image-registry-cd96cb6c-kpns4\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:58.752152 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.752112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" event={"ID":"6360c550-5517-46fc-adbb-5730bf3a31b8","Type":"ContainerStarted","Data":"c1c5e5b4096aa6bb03ac3db914d9cf43bd4a6c581bfc89c8fa16e88ee1e230d0"} Jul 18 11:49:58.878241 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.878204 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g4x2w\"" Jul 18 11:49:58.885590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.885324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:58.900888 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.900863 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qxztp\"" Jul 18 11:49:58.908896 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:58.908864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mm94p" Jul 18 11:49:59.050466 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.049861 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:49:59.056495 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:59.056455 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2aa38c0_d758_49c1_ad20_3a614100e76c.slice/crio-6f0da406eac78127b8ba9542e338ff5c3bd0feb8924c972720066356539461ed WatchSource:0}: Error finding container 6f0da406eac78127b8ba9542e338ff5c3bd0feb8924c972720066356539461ed: Status 404 returned error can't find the container with id 6f0da406eac78127b8ba9542e338ff5c3bd0feb8924c972720066356539461ed Jul 18 11:49:59.068610 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.068525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mm94p"] Jul 18 11:49:59.072220 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:49:59.072185 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode156db75_f982_4a5e_8cbe_8b06b24aa23f.slice/crio-1f4c93ed74b9035410a54e94824f6f81960fb85bad250f43b896905a733be428 WatchSource:0}: Error finding container 1f4c93ed74b9035410a54e94824f6f81960fb85bad250f43b896905a733be428: Status 404 returned error can't find the container with id 1f4c93ed74b9035410a54e94824f6f81960fb85bad250f43b896905a733be428 Jul 18 11:49:59.760510 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.760474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" event={"ID":"c2aa38c0-d758-49c1-ad20-3a614100e76c","Type":"ContainerStarted","Data":"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a"} Jul 18 11:49:59.760942 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.760563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" event={"ID":"c2aa38c0-d758-49c1-ad20-3a614100e76c","Type":"ContainerStarted","Data":"6f0da406eac78127b8ba9542e338ff5c3bd0feb8924c972720066356539461ed"} Jul 18 11:49:59.760942 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.760592 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:49:59.761886 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.761856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mm94p" event={"ID":"e156db75-f982-4a5e-8cbe-8b06b24aa23f","Type":"ContainerStarted","Data":"1f4c93ed74b9035410a54e94824f6f81960fb85bad250f43b896905a733be428"} Jul 18 11:49:59.778988 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:49:59.778943 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" podStartSLOduration=68.778930619 podStartE2EDuration="1m8.778930619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:49:59.777809965 +0000 UTC m=+68.849869494" watchObservedRunningTime="2026-07-18 11:49:59.778930619 +0000 UTC m=+68.850990121" Jul 18 11:50:01.769882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:01.769847 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" event={"ID":"6360c550-5517-46fc-adbb-5730bf3a31b8","Type":"ContainerStarted","Data":"a905a0b6a59e94691b107a2058831bc86a09c900ab39e4e9009c5a0d57b44a21"} Jul 18 11:50:01.770392 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:01.769904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" event={"ID":"6360c550-5517-46fc-adbb-5730bf3a31b8","Type":"ContainerStarted","Data":"53b534469682a556832960b230c00cd2c68e2717cb5a0e56154361216b2c7c53"} Jul 18 11:50:01.771504 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:01.771476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ndsm5" event={"ID":"c73af55a-7e9a-416b-8144-38477c907ba0","Type":"ContainerStarted","Data":"c5a1e4bbb2881a13a708152ecbffe3a1e651b04dfebc9ca9d85ec5ffd9a87b75"} Jul 18 11:50:01.784604 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:01.784552 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6r88v" podStartSLOduration=17.453128004 podStartE2EDuration="20.784532182s" totalImagesPullingTime="3.331404178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:01.784115681 +0000 UTC m=+70.856175208" watchObservedRunningTime="2026-07-18 11:50:01.784532182 +0000 UTC m=+70.856591686" Jul 18 11:50:01.798794 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:01.798737 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ndsm5" podStartSLOduration=66.244007133 podStartE2EDuration="1m10.798722717s" totalImagesPullingTime="4.554715584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:01.797991028 +0000 UTC m=+70.870050534" watchObservedRunningTime="2026-07-18 11:50:01.798722717 +0000 UTC m=+70.870782198" Jul 18 11:50:02.570579 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.570486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:02.570579 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.570575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:02.570775 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:02.570636 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 11:50:02.570775 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:02.570703 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls podName:9d3ecbf5-f159-4b01-a1f1-8262a78d6b81 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:18.570687531 +0000 UTC m=+87.642747012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls") pod "insights-runtime-extractor-psvl5" (UID: "9d3ecbf5-f159-4b01-a1f1-8262a78d6b81") : secret "insights-runtime-extractor-tls" not found Jul 18 11:50:02.571187 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.571170 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:02.776711 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.776679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mm94p" event={"ID":"e156db75-f982-4a5e-8cbe-8b06b24aa23f","Type":"ContainerStarted","Data":"8f1f4a25cd8893c7cc339d0ed9277714d16d2414513da44173d70687d6f5e763"} Jul 18 11:50:02.777197 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.776761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mm94p" event={"ID":"e156db75-f982-4a5e-8cbe-8b06b24aa23f","Type":"ContainerStarted","Data":"21391b9884d2d669189725a33f797f56f8bcdb7c9ff191f3e5532647cb2d1494"} Jul 18 11:50:02.821475 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:02.821360 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mm94p" podStartSLOduration=33.604413172 podStartE2EDuration="36.82134295s" totalImagesPullingTime="3.216929778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:02.821195047 +0000 UTC m=+71.893254561" watchObservedRunningTime="2026-07-18 11:50:02.82134295 +0000 UTC m=+71.893402470" Jul 18 11:50:03.780846 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:03.780815 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mm94p" Jul 18 11:50:12.223856 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.223825 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7"] Jul 18 11:50:12.226694 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.226678 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.229688 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.229666 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 11:50:12.230628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.230506 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 11:50:12.230628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.230573 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 11:50:12.232278 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.232264 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 18 11:50:12.252278 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.252255 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7"] Jul 18 11:50:12.346040 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.346006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18604306-0ac4-412d-8166-8fdbba880120-tmp\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.346226 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.346057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmhk\" (UniqueName: \"kubernetes.io/projected/18604306-0ac4-412d-8166-8fdbba880120-kube-api-access-5hmhk\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.346226 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.346126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18604306-0ac4-412d-8166-8fdbba880120-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.446638 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.446606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18604306-0ac4-412d-8166-8fdbba880120-tmp\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.446828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.446643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmhk\" (UniqueName: \"kubernetes.io/projected/18604306-0ac4-412d-8166-8fdbba880120-kube-api-access-5hmhk\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.446828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.446671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18604306-0ac4-412d-8166-8fdbba880120-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.447088 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.447065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18604306-0ac4-412d-8166-8fdbba880120-tmp\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.449254 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.449227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18604306-0ac4-412d-8166-8fdbba880120-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.462509 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.462485 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmhk\" (UniqueName: \"kubernetes.io/projected/18604306-0ac4-412d-8166-8fdbba880120-kube-api-access-5hmhk\") pod \"klusterlet-addon-workmgr-7bbf77b6ff-dn6v7\" (UID: \"18604306-0ac4-412d-8166-8fdbba880120\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.536104 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.536027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:12.656038 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.656005 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7"] Jul 18 11:50:12.659829 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:12.659804 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18604306_0ac4_412d_8166_8fdbba880120.slice/crio-eb4208002eb9fd70652f1c67c81ab3c17a5ff1da01133927360497d4e8ea1915 WatchSource:0}: Error finding container eb4208002eb9fd70652f1c67c81ab3c17a5ff1da01133927360497d4e8ea1915: Status 404 returned error can't find the container with id eb4208002eb9fd70652f1c67c81ab3c17a5ff1da01133927360497d4e8ea1915 Jul 18 11:50:12.813030 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:12.812949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" event={"ID":"18604306-0ac4-412d-8166-8fdbba880120","Type":"ContainerStarted","Data":"eb4208002eb9fd70652f1c67c81ab3c17a5ff1da01133927360497d4e8ea1915"} Jul 18 11:50:13.554740 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.554694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:50:13.555246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.554749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:50:13.557872 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.557846 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2fmbx\" (UID: \"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:50:13.557971 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.557913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5d2213a6-b613-4961-9d5d-6b483a05b011-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-dc5cb\" (UID: \"5d2213a6-b613-4961-9d5d-6b483a05b011\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:50:13.786207 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.786086 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mm94p" Jul 18 11:50:13.793805 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.793774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dnb2b\"" Jul 18 11:50:13.802255 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.802122 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" Jul 18 11:50:13.806863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.806841 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cjg2l\"" Jul 18 11:50:13.824993 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.814719 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" Jul 18 11:50:13.964443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.964412 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb"] Jul 18 11:50:13.968860 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:13.968831 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2213a6_b613_4961_9d5d_6b483a05b011.slice/crio-6ae6f77f887b979a3f107aeb17b7d45cde3c880f7647cb793d009829b0aabe54 WatchSource:0}: Error finding container 6ae6f77f887b979a3f107aeb17b7d45cde3c880f7647cb793d009829b0aabe54: Status 404 returned error can't find the container with id 6ae6f77f887b979a3f107aeb17b7d45cde3c880f7647cb793d009829b0aabe54 Jul 18 11:50:13.983329 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:13.982276 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx"] Jul 18 11:50:13.987288 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:13.987258 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5caa8a_0f0d_4b5a_a096_2a7c12d5a4a9.slice/crio-7a1a6018c1e0c23322261bf937238fe5f89495f841be635d865b6c33ccf4b913 WatchSource:0}: Error finding container 7a1a6018c1e0c23322261bf937238fe5f89495f841be635d865b6c33ccf4b913: Status 404 returned error can't find the container with id 7a1a6018c1e0c23322261bf937238fe5f89495f841be635d865b6c33ccf4b913 Jul 18 11:50:14.831033 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:14.830991 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" event={"ID":"5d2213a6-b613-4961-9d5d-6b483a05b011","Type":"ContainerStarted","Data":"6ae6f77f887b979a3f107aeb17b7d45cde3c880f7647cb793d009829b0aabe54"} Jul 18 11:50:14.832312 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:14.832279 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" event={"ID":"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9","Type":"ContainerStarted","Data":"7a1a6018c1e0c23322261bf937238fe5f89495f841be635d865b6c33ccf4b913"} Jul 18 11:50:16.112321 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.112291 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:50:16.127294 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.127263 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:50:16.127427 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.127384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.130125 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.129951 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 11:50:16.130668 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 11:50:16.130668 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 11:50:16.130858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130701 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 11:50:16.130858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130741 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p65kk\"" Jul 18 11:50:16.130858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130642 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 11:50:16.130858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.130785 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 11:50:16.131118 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.131086 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 11:50:16.275666 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.275666 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.275930 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.275930 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.275930 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.276137 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.275970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q7q8\" (UniqueName: \"kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377154 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q7q8\" (UniqueName: \"kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377154 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377154 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377217 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.377443 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.378024 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.377998 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.378154 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.378010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.378154 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.378067 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.380412 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.380392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.380511 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.380482 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.385279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.385255 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q7q8\" (UniqueName: \"kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8\") pod \"console-5858dff758-f4lpv\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:16.439863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:16.439828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:17.388951 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.388482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:50:17.394619 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:17.394595 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d0a9f7_8e19_4b97_9545_56d7d94909f2.slice/crio-b803fa1db0f92fdfaee934a53407e57745bbfc4ce35810fa4d7f3694c4b70d3c WatchSource:0}: Error finding container b803fa1db0f92fdfaee934a53407e57745bbfc4ce35810fa4d7f3694c4b70d3c: Status 404 returned error can't find the container with id b803fa1db0f92fdfaee934a53407e57745bbfc4ce35810fa4d7f3694c4b70d3c Jul 18 11:50:17.735434 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.735402 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz"] Jul 18 11:50:17.738451 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.738431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:17.740804 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.740773 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 18 11:50:17.740949 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.740932 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6s4br\"" Jul 18 11:50:17.749455 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.749427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz"] Jul 18 11:50:17.843955 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.843917 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" event={"ID":"5d2213a6-b613-4961-9d5d-6b483a05b011","Type":"ContainerStarted","Data":"2e9ff27c3c089df6ff7217e418dfc03ddef1081455cc68a9c599a12a4f7a7be2"} Jul 18 11:50:17.845638 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.845607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" event={"ID":"18604306-0ac4-412d-8166-8fdbba880120","Type":"ContainerStarted","Data":"67da75ef30247264f227c85202c62895ea5a10ad504a61f20d89279689a28f06"} Jul 18 11:50:17.845768 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.845743 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:17.847481 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.847418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" event={"ID":"3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9","Type":"ContainerStarted","Data":"f6b411d92f0227dd7734ebdea4e537c62ead7ef91285739bbbef7f3fddbea031"} Jul 18 11:50:17.847957 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.847935 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" Jul 18 11:50:17.848718 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.848698 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858dff758-f4lpv" event={"ID":"76d0a9f7-8e19-4b97-9545-56d7d94909f2","Type":"ContainerStarted","Data":"b803fa1db0f92fdfaee934a53407e57745bbfc4ce35810fa4d7f3694c4b70d3c"} Jul 18 11:50:17.861296 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.861258 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-dc5cb" podStartSLOduration=33.583345714000004 podStartE2EDuration="36.861245953s" totalImagesPullingTime="3.277900239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:17.860578127 +0000 UTC m=+86.932637628" watchObservedRunningTime="2026-07-18 11:50:17.861245953 +0000 UTC m=+86.933305456" Jul 18 11:50:17.874431 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.874387 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2fmbx" podStartSLOduration=33.611070084 podStartE2EDuration="36.874370769s" totalImagesPullingTime="3.263300685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:17.873793257 +0000 UTC m=+86.945852764" watchObservedRunningTime="2026-07-18 11:50:17.874370769 +0000 UTC m=+86.946430274" Jul 18 11:50:17.890999 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.890897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dlmbz\" (UID: \"adc5ba80-f709-4803-8e30-c6a1828ef6eb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:17.892825 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.892782 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bbf77b6ff-dn6v7" podStartSLOduration=1.305359521 podStartE2EDuration="5.89277029s" totalImagesPullingTime="4.587410769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:17.892517128 +0000 UTC m=+86.964576634" watchObservedRunningTime="2026-07-18 11:50:17.89277029 +0000 UTC m=+86.964829787" Jul 18 11:50:17.991642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:17.991547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dlmbz\" (UID: \"adc5ba80-f709-4803-8e30-c6a1828ef6eb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:17.992328 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:17.991926 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 18 11:50:17.992328 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:17.992006 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates podName:adc5ba80-f709-4803-8e30-c6a1828ef6eb nodeName:}" failed. No retries permitted until 2026-07-18 11:50:18.491983775 +0000 UTC m=+87.564043259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-dlmbz" (UID: "adc5ba80-f709-4803-8e30-c6a1828ef6eb") : secret "prometheus-operator-admission-webhook-tls" not found Jul 18 11:50:18.497343 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.497295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dlmbz\" (UID: \"adc5ba80-f709-4803-8e30-c6a1828ef6eb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:18.500016 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.499988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/adc5ba80-f709-4803-8e30-c6a1828ef6eb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dlmbz\" (UID: \"adc5ba80-f709-4803-8e30-c6a1828ef6eb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:18.598490 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.598451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:18.601133 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.601107 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d3ecbf5-f159-4b01-a1f1-8262a78d6b81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-psvl5\" (UID: \"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81\") " pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:18.649723 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.649695 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:18.786248 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.786035 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz"] Jul 18 11:50:18.788914 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:18.788882 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc5ba80_f709_4803_8e30_c6a1828ef6eb.slice/crio-21e92eff71d91d9afd38f0e9094455bd405a5558e8fc1c9126351b62095fe524 WatchSource:0}: Error finding container 21e92eff71d91d9afd38f0e9094455bd405a5558e8fc1c9126351b62095fe524: Status 404 returned error can't find the container with id 21e92eff71d91d9afd38f0e9094455bd405a5558e8fc1c9126351b62095fe524 Jul 18 11:50:18.853560 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.853527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" event={"ID":"adc5ba80-f709-4803-8e30-c6a1828ef6eb","Type":"ContainerStarted","Data":"21e92eff71d91d9afd38f0e9094455bd405a5558e8fc1c9126351b62095fe524"} Jul 18 11:50:18.890792 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.890741 2578 patch_prober.go:28] interesting pod/image-registry-cd96cb6c-kpns4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 18 11:50:18.890975 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.890816 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 11:50:18.890975 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.890837 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-29g54\"" Jul 18 11:50:18.898916 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:18.898897 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-psvl5" Jul 18 11:50:19.041965 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:19.041945 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-psvl5"] Jul 18 11:50:19.044525 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:19.044498 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3ecbf5_f159_4b01_a1f1_8262a78d6b81.slice/crio-7f0b25d203ccc1622f18eb127a250f91e5890d0db0c81efd893f9e0890dd9a06 WatchSource:0}: Error finding container 7f0b25d203ccc1622f18eb127a250f91e5890d0db0c81efd893f9e0890dd9a06: Status 404 returned error can't find the container with id 7f0b25d203ccc1622f18eb127a250f91e5890d0db0c81efd893f9e0890dd9a06 Jul 18 11:50:19.859103 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:19.859062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-psvl5" event={"ID":"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81","Type":"ContainerStarted","Data":"7f0b25d203ccc1622f18eb127a250f91e5890d0db0c81efd893f9e0890dd9a06"} Jul 18 11:50:20.771624 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.771542 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:50:20.865382 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.865337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" event={"ID":"adc5ba80-f709-4803-8e30-c6a1828ef6eb","Type":"ContainerStarted","Data":"80def30c5da57f5b47b6128635be7f190200d261905df1c7e4b0c7275e56f2d9"} Jul 18 11:50:20.865817 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.865525 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:20.867592 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.867566 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858dff758-f4lpv" event={"ID":"76d0a9f7-8e19-4b97-9545-56d7d94909f2","Type":"ContainerStarted","Data":"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59"} Jul 18 11:50:20.871289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.871266 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" Jul 18 11:50:20.882275 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.882234 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dlmbz" podStartSLOduration=2.167497464 podStartE2EDuration="3.882220285s" totalImagesPullingTime="1.714722821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:20.880709256 +0000 UTC m=+89.952768810" watchObservedRunningTime="2026-07-18 11:50:20.882220285 +0000 UTC m=+89.954279766" Jul 18 11:50:20.911476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:20.911423 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5858dff758-f4lpv" podStartSLOduration=1.801337285 podStartE2EDuration="4.911402906s" totalImagesPullingTime="3.110065621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:20.910406323 +0000 UTC m=+89.982465849" watchObservedRunningTime="2026-07-18 11:50:20.911402906 +0000 UTC m=+89.983462411" Jul 18 11:50:21.808418 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.808387 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk"] Jul 18 11:50:21.811828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.811806 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:21.815143 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.814989 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 11:50:21.815143 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.815017 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bmzbj\"" Jul 18 11:50:21.815143 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.815080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 18 11:50:21.815143 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.815017 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 18 11:50:21.819455 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.819423 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk"] Jul 18 11:50:21.926698 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.926661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlztr\" (UniqueName: \"kubernetes.io/projected/c290effa-8784-4b77-8755-482d2cde809c-kube-api-access-jlztr\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:21.927183 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.926704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c290effa-8784-4b77-8755-482d2cde809c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:21.927183 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.926812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:21.927183 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:21.926895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.027849 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.027800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlztr\" (UniqueName: \"kubernetes.io/projected/c290effa-8784-4b77-8755-482d2cde809c-kube-api-access-jlztr\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.027849 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.027854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c290effa-8784-4b77-8755-482d2cde809c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.028137 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.027926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.028137 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.028090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.029109 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.029082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c290effa-8784-4b77-8755-482d2cde809c-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.030897 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.030868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.031061 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.031028 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c290effa-8784-4b77-8755-482d2cde809c-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.035956 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.035927 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlztr\" (UniqueName: \"kubernetes.io/projected/c290effa-8784-4b77-8755-482d2cde809c-kube-api-access-jlztr\") pod \"prometheus-operator-5945b5cfd-zbvsk\" (UID: \"c290effa-8784-4b77-8755-482d2cde809c\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.122868 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.122786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" Jul 18 11:50:22.267242 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.267203 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk"] Jul 18 11:50:22.269032 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:22.269009 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc290effa_8784_4b77_8755_482d2cde809c.slice/crio-a818704ed28d0d8f1a8e7a10a8bca13dec2c2bb98f5d55e6ea4b84d7ea20eaad WatchSource:0}: Error finding container a818704ed28d0d8f1a8e7a10a8bca13dec2c2bb98f5d55e6ea4b84d7ea20eaad: Status 404 returned error can't find the container with id a818704ed28d0d8f1a8e7a10a8bca13dec2c2bb98f5d55e6ea4b84d7ea20eaad Jul 18 11:50:22.877788 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:22.877740 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" event={"ID":"c290effa-8784-4b77-8755-482d2cde809c","Type":"ContainerStarted","Data":"a818704ed28d0d8f1a8e7a10a8bca13dec2c2bb98f5d55e6ea4b84d7ea20eaad"} Jul 18 11:50:25.093033 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.092999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:50:25.096703 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.096679 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.105099 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.105071 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 11:50:25.109073 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.109037 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:50:25.155256 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155602 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2jf8\" (UniqueName: \"kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.155602 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.155488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.256646 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.256828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2jf8\" (UniqueName: \"kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.256828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.256828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.256954 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257001 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.256970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257072 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.257012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257537 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.257477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257702 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.257681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257781 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.257681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.257850 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.257829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.259531 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.259510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.259990 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.259968 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.266734 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.266712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2jf8\" (UniqueName: \"kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8\") pod \"console-5f85d96bc-8l4hf\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.407829 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.407802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:25.546666 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.546622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:50:25.550578 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:25.550546 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c58ba2_c338_45b3_aec0_54e0b4d45c92.slice/crio-0a298950a4b7803942ae3b808e23280391c7585ccfb7ebbcef86be841f2bfb9a WatchSource:0}: Error finding container 0a298950a4b7803942ae3b808e23280391c7585ccfb7ebbcef86be841f2bfb9a: Status 404 returned error can't find the container with id 0a298950a4b7803942ae3b808e23280391c7585ccfb7ebbcef86be841f2bfb9a Jul 18 11:50:25.893279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.893188 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" event={"ID":"c290effa-8784-4b77-8755-482d2cde809c","Type":"ContainerStarted","Data":"43f8f00680ee0fa25974417bd16b9de204e94e9920c930693aab4a60c85f2979"} Jul 18 11:50:25.893279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.893257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" event={"ID":"c290effa-8784-4b77-8755-482d2cde809c","Type":"ContainerStarted","Data":"2fd70c1ced13a7476e0f02e900a0739c80d36b9b704c833e625f1f90cc219bde"} Jul 18 11:50:25.895206 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.895175 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f85d96bc-8l4hf" event={"ID":"89c58ba2-c338-45b3-aec0-54e0b4d45c92","Type":"ContainerStarted","Data":"bda8af5265d028d3c1f4405a6e21eadeeb0f58c83343de7ca1e0e1434e56bc1b"} Jul 18 11:50:25.895330 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.895228 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f85d96bc-8l4hf" event={"ID":"89c58ba2-c338-45b3-aec0-54e0b4d45c92","Type":"ContainerStarted","Data":"0a298950a4b7803942ae3b808e23280391c7585ccfb7ebbcef86be841f2bfb9a"} Jul 18 11:50:25.896768 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.896746 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-psvl5" event={"ID":"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81","Type":"ContainerStarted","Data":"f2800e73e7aa5b4cb4e8b536235766fb4d1fa711ded9ec62c0edca7fd4b22b4f"} Jul 18 11:50:25.910601 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.909829 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-zbvsk" podStartSLOduration=1.805748028 podStartE2EDuration="4.909810509s" totalImagesPullingTime="3.104062481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:25.909026581 +0000 UTC m=+94.981086085" watchObservedRunningTime="2026-07-18 11:50:25.909810509 +0000 UTC m=+94.981869999" Jul 18 11:50:25.930120 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:25.929923 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f85d96bc-8l4hf" podStartSLOduration=0.929905568 podStartE2EDuration="929.905568ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:25.928137816 +0000 UTC m=+95.000197334" watchObservedRunningTime="2026-07-18 11:50:25.929905568 +0000 UTC m=+95.001965071" Jul 18 11:50:26.440527 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:26.440345 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:26.440527 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:26.440411 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:26.447544 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:26.447518 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:26.904729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:26.904621 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:50:27.751369 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:27.751336 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-998xh" Jul 18 11:50:28.258742 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.258511 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2k5m2"] Jul 18 11:50:28.264098 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.264073 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.267965 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.267920 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 11:50:28.268129 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.267981 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-grnx4\"" Jul 18 11:50:28.268355 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.268339 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 11:50:28.268528 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.268497 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 11:50:28.280832 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.280807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.280938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.280863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.280938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.280891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.280938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.280914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhczz\" (UniqueName: \"kubernetes.io/projected/c6a38575-245e-4416-9f8d-b904b2fa815e-kube-api-access-rhczz\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.281081 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.281026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-textfile\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.281122 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.281077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-sys\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.281122 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.281096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-wtmp\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.281200 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.281124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-metrics-client-ca\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.281200 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.281152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-root\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.382575 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.382575 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhczz\" (UniqueName: \"kubernetes.io/projected/c6a38575-245e-4416-9f8d-b904b2fa815e-kube-api-access-rhczz\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-textfile\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:28.382704 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:28.382779 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls podName:c6a38575-245e-4416-9f8d-b904b2fa815e nodeName:}" failed. No retries permitted until 2026-07-18 11:50:28.882759907 +0000 UTC m=+97.954819391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls") pod "node-exporter-2k5m2" (UID: "c6a38575-245e-4416-9f8d-b904b2fa815e") : secret "node-exporter-tls" not found Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-sys\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-wtmp\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-metrics-client-ca\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.382925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-root\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-wtmp\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-textfile\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-sys\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.383385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6a38575-245e-4416-9f8d-b904b2fa815e-root\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.384193 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383691 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-metrics-client-ca\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.384193 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.383851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.385956 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.385934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.394541 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.394484 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhczz\" (UniqueName: \"kubernetes.io/projected/c6a38575-245e-4416-9f8d-b904b2fa815e-kube-api-access-rhczz\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.887262 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:28.887232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:28.887662 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:28.887375 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 11:50:28.887662 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:28.887439 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls podName:c6a38575-245e-4416-9f8d-b904b2fa815e nodeName:}" failed. No retries permitted until 2026-07-18 11:50:29.887424388 +0000 UTC m=+98.959483870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls") pod "node-exporter-2k5m2" (UID: "c6a38575-245e-4416-9f8d-b904b2fa815e") : secret "node-exporter-tls" not found Jul 18 11:50:29.899195 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:29.899140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:29.901829 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:29.901803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6a38575-245e-4416-9f8d-b904b2fa815e-node-exporter-tls\") pod \"node-exporter-2k5m2\" (UID: \"c6a38575-245e-4416-9f8d-b904b2fa815e\") " pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:29.914877 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:29.914850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-psvl5" event={"ID":"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81","Type":"ContainerStarted","Data":"7675313b1dd6c8161189672d546696e71d871ea1ee42aa3431da0c0e6c84267f"} Jul 18 11:50:30.075612 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.075571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2k5m2" Jul 18 11:50:30.087778 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:30.087746 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a38575_245e_4416_9f8d_b904b2fa815e.slice/crio-d225f727f616ea79a7195f95913652ea7b7e6c089a3aaf25418e0a9ae5210c83 WatchSource:0}: Error finding container d225f727f616ea79a7195f95913652ea7b7e6c089a3aaf25418e0a9ae5210c83: Status 404 returned error can't find the container with id d225f727f616ea79a7195f95913652ea7b7e6c089a3aaf25418e0a9ae5210c83 Jul 18 11:50:30.202778 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.202743 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59bf87698d-rdrvx"] Jul 18 11:50:30.207652 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.207633 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.210669 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.210522 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 18 11:50:30.210669 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.210527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 18 11:50:30.210669 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.210527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 18 11:50:30.210990 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.210969 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 18 11:50:30.211109 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.211080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g4fhz\"" Jul 18 11:50:30.211247 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.211231 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-adddc6ajpqtl3\"" Jul 18 11:50:30.211398 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.211371 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 18 11:50:30.219176 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.219114 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59bf87698d-rdrvx"] Jul 18 11:50:30.303424 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303395 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrs68\" (UniqueName: \"kubernetes.io/projected/12707f1a-abc1-4108-8ff2-45ee0879c600-kube-api-access-xrs68\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303581 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303581 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-grpc-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303581 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303727 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303727 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12707f1a-abc1-4108-8ff2-45ee0879c600-metrics-client-ca\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303727 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.303846 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.303814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.404942 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.404911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.404953 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrs68\" (UniqueName: \"kubernetes.io/projected/12707f1a-abc1-4108-8ff2-45ee0879c600-kube-api-access-xrs68\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.404975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.404995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-grpc-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.405034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.405096 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405130 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.405122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12707f1a-abc1-4108-8ff2-45ee0879c600-metrics-client-ca\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.405447 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.405178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.406290 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.406242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12707f1a-abc1-4108-8ff2-45ee0879c600-metrics-client-ca\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.408303 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.408277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.408303 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.408291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.408918 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.408728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.409026 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.408845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.409026 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.408876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-grpc-tls\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.409026 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.409006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/12707f1a-abc1-4108-8ff2-45ee0879c600-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.415605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.415583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrs68\" (UniqueName: \"kubernetes.io/projected/12707f1a-abc1-4108-8ff2-45ee0879c600-kube-api-access-xrs68\") pod \"thanos-querier-59bf87698d-rdrvx\" (UID: \"12707f1a-abc1-4108-8ff2-45ee0879c600\") " pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.520075 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.519941 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:30.685031 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.684995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59bf87698d-rdrvx"] Jul 18 11:50:30.689003 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:30.688969 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12707f1a_abc1_4108_8ff2_45ee0879c600.slice/crio-8f67a5c3ba6282aa3eb6fce18282ab5c30370a0121b71606a127e154c3effa47 WatchSource:0}: Error finding container 8f67a5c3ba6282aa3eb6fce18282ab5c30370a0121b71606a127e154c3effa47: Status 404 returned error can't find the container with id 8f67a5c3ba6282aa3eb6fce18282ab5c30370a0121b71606a127e154c3effa47 Jul 18 11:50:30.924474 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.924440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k5m2" event={"ID":"c6a38575-245e-4416-9f8d-b904b2fa815e","Type":"ContainerStarted","Data":"d225f727f616ea79a7195f95913652ea7b7e6c089a3aaf25418e0a9ae5210c83"} Jul 18 11:50:30.925654 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:30.925631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"8f67a5c3ba6282aa3eb6fce18282ab5c30370a0121b71606a127e154c3effa47"} Jul 18 11:50:32.920393 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:32.920361 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk"] Jul 18 11:50:32.938855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:32.938821 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk"] Jul 18 11:50:32.939007 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:32.938946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:32.941081 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:32.941060 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 18 11:50:32.941203 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:32.941121 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nqbf8\"" Jul 18 11:50:33.030351 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.030309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9gtm\" (UniqueName: \"kubernetes.io/projected/58a2b2fa-c547-49e8-b2a4-dc82281374a8-kube-api-access-n9gtm\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.030524 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.030422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.131615 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.131575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.131791 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.131638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gtm\" (UniqueName: \"kubernetes.io/projected/58a2b2fa-c547-49e8-b2a4-dc82281374a8-kube-api-access-n9gtm\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.131791 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:33.131725 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 18 11:50:33.131791 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:50:33.131783 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert podName:58a2b2fa-c547-49e8-b2a4-dc82281374a8 nodeName:}" failed. No retries permitted until 2026-07-18 11:50:33.63176847 +0000 UTC m=+102.703827951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-w68zk" (UID: "58a2b2fa-c547-49e8-b2a4-dc82281374a8") : secret "monitoring-plugin-cert" not found Jul 18 11:50:33.142235 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.142213 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9gtm\" (UniqueName: \"kubernetes.io/projected/58a2b2fa-c547-49e8-b2a4-dc82281374a8-kube-api-access-n9gtm\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.312461 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.312379 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:50:33.341218 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.341191 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:50:33.356873 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.356850 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:50:33.356984 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.356934 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.434856 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.434828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435012 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.434872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435012 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.434888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435012 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.434984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435012 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.435009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435185 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.435062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.435185 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.435093 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nql46\" (UniqueName: \"kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.536479 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.536441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.536677 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.536656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.536755 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.536701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.536817 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.536754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.536817 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.536777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.538545 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.538029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.538545 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.538127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nql46\" (UniqueName: \"kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.538545 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.538194 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.538545 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.538386 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.538806 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.538620 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.540629 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.540039 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.540629 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.540439 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.540629 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.540491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.548174 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.548154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nql46\" (UniqueName: \"kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46\") pod \"console-66f8c65c4-tsg7s\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.640551 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.640524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.643177 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.643150 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58a2b2fa-c547-49e8-b2a4-dc82281374a8-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-w68zk\" (UID: \"58a2b2fa-c547-49e8-b2a4-dc82281374a8\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.665963 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.665938 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:33.851418 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.851394 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:33.855420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.855392 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:50:33.907149 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:33.907110 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e9ad3a_2e7f_4289_8df2_5e5c3f4866b0.slice/crio-05c927dc7e8a4fa8d1d89fecd9b1c677c064b13b869d1ee04af088ebc1c937ef WatchSource:0}: Error finding container 05c927dc7e8a4fa8d1d89fecd9b1c677c064b13b869d1ee04af088ebc1c937ef: Status 404 returned error can't find the container with id 05c927dc7e8a4fa8d1d89fecd9b1c677c064b13b869d1ee04af088ebc1c937ef Jul 18 11:50:33.943447 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.943413 2578 generic.go:356] "Generic (PLEG): container finished" podID="c6a38575-245e-4416-9f8d-b904b2fa815e" containerID="3ff14b925821dfa350c5bbd5405782b9716c07938f43ce8a04b3f4e5531c2981" exitCode=0 Jul 18 11:50:33.944302 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.943521 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k5m2" event={"ID":"c6a38575-245e-4416-9f8d-b904b2fa815e","Type":"ContainerDied","Data":"3ff14b925821dfa350c5bbd5405782b9716c07938f43ce8a04b3f4e5531c2981"} Jul 18 11:50:33.947517 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.946892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-psvl5" event={"ID":"9d3ecbf5-f159-4b01-a1f1-8262a78d6b81","Type":"ContainerStarted","Data":"b9f8cac549ad84bac303f3affae9082ebb578060eb0a43cfcc9bcba07923fb2c"} Jul 18 11:50:33.950097 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.949844 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f8c65c4-tsg7s" event={"ID":"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0","Type":"ContainerStarted","Data":"05c927dc7e8a4fa8d1d89fecd9b1c677c064b13b869d1ee04af088ebc1c937ef"} Jul 18 11:50:33.977586 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:33.977232 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-psvl5" podStartSLOduration=33.578464771 podStartE2EDuration="47.977210637s" totalImagesPullingTime="14.398745866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 11:49:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:33.975669346 +0000 UTC m=+103.047728867" watchObservedRunningTime="2026-07-18 11:50:33.977210637 +0000 UTC m=+103.049270143" Jul 18 11:50:34.018734 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.018710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk"] Jul 18 11:50:34.021342 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:50:34.021313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a2b2fa_c547_49e8_b2a4_dc82281374a8.slice/crio-932255c02d4772cf1abdcc197eb5f8b75df65ed55056ec49d2a2bd1324b3a34a WatchSource:0}: Error finding container 932255c02d4772cf1abdcc197eb5f8b75df65ed55056ec49d2a2bd1324b3a34a: Status 404 returned error can't find the container with id 932255c02d4772cf1abdcc197eb5f8b75df65ed55056ec49d2a2bd1324b3a34a Jul 18 11:50:34.956184 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.956141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" event={"ID":"58a2b2fa-c547-49e8-b2a4-dc82281374a8","Type":"ContainerStarted","Data":"932255c02d4772cf1abdcc197eb5f8b75df65ed55056ec49d2a2bd1324b3a34a"} Jul 18 11:50:34.959421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.959327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k5m2" event={"ID":"c6a38575-245e-4416-9f8d-b904b2fa815e","Type":"ContainerStarted","Data":"7cbb9a84a7e04401c48340ef5be397730fd5351250975bd3bbe4978ada27d799"} Jul 18 11:50:34.959421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.959383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2k5m2" event={"ID":"c6a38575-245e-4416-9f8d-b904b2fa815e","Type":"ContainerStarted","Data":"6f4c7b42bea0069a8dc176a2c69173cb16462c1c09e2b33c60828982b90cfed9"} Jul 18 11:50:34.961943 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.961913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f8c65c4-tsg7s" event={"ID":"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0","Type":"ContainerStarted","Data":"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f"} Jul 18 11:50:34.981683 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.981630 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2k5m2" podStartSLOduration=3.3891249979999998 podStartE2EDuration="6.981613147s" totalImagesPullingTime="3.49652673s" totalInitContainerRuntime="95.961419ms" isStatefulPod=true podCreationTimestamp="2026-07-18 11:50:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:34.980248657 +0000 UTC m=+104.052308173" watchObservedRunningTime="2026-07-18 11:50:34.981613147 +0000 UTC m=+104.053672651" Jul 18 11:50:34.984867 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:34.984844 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:50:35.023437 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.023371 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66f8c65c4-tsg7s" podStartSLOduration=2.023351652 podStartE2EDuration="2.023351652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:35.022249654 +0000 UTC m=+104.094309171" watchObservedRunningTime="2026-07-18 11:50:35.023351652 +0000 UTC m=+104.095411157" Jul 18 11:50:35.414076 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.413965 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:35.966987 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.966904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" event={"ID":"58a2b2fa-c547-49e8-b2a4-dc82281374a8","Type":"ContainerStarted","Data":"6dccc32c4c9482b7a08bd720b9fcc1e94dc83ac6d849e977e231b214475b0317"} Jul 18 11:50:35.967445 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.967109 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:35.969340 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.969313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"651a86b61ee4ef60e836bbb4d53bf001e291d61be5b723f7b14d89dca9a810b1"} Jul 18 11:50:35.969457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.969366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"e05e1cf30b134c94ffb55de5e100ca384ddb2ffd6a85dbc1123b9ea020638554"} Jul 18 11:50:35.969457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.969401 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"84809271a9e613564895f35d154818b565fc6d505caedaae2463ee30e029867d"} Jul 18 11:50:35.972662 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.972644 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" Jul 18 11:50:35.983466 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:35.983430 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-w68zk" podStartSLOduration=2.2896385 podStartE2EDuration="3.983416933s" totalImagesPullingTime="1.693778433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:35.98222132 +0000 UTC m=+105.054280825" watchObservedRunningTime="2026-07-18 11:50:35.983416933 +0000 UTC m=+105.055476414" Jul 18 11:50:36.976710 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:36.976673 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"bf13370dbf6b2fe22bc42f5bfebedb5d666c40d0de0e2753c5ec0bab5d4557bb"} Jul 18 11:50:36.977094 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:36.976747 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"2966ff953b27ae76e1080076e98932be1b49bfe33b6ee65d23fddde1624d65ab"} Jul 18 11:50:36.977094 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:36.976781 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" event={"ID":"12707f1a-abc1-4108-8ff2-45ee0879c600","Type":"ContainerStarted","Data":"b6eec45e5312df6e54dbdaffdaf49c81a98b0da8b8fc83bfad98ef03f171e775"} Jul 18 11:50:36.999102 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:36.999033 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" podStartSLOduration=1.110523263 podStartE2EDuration="6.999019132s" totalImagesPullingTime="5.888495869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:50:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:50:36.998694212 +0000 UTC m=+106.070753726" watchObservedRunningTime="2026-07-18 11:50:36.999019132 +0000 UTC m=+106.071078634" Jul 18 11:50:37.980961 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:37.980931 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:43.666967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:43.666924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:43.667422 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:43.666986 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:43.671623 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:43.671601 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:43.993122 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:43.993089 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59bf87698d-rdrvx" Jul 18 11:50:44.007081 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:44.007034 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:50:44.054203 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:44.054170 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:50:58.332457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:58.332399 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f85d96bc-8l4hf" podUID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" containerName="console" containerID="cri-o://bda8af5265d028d3c1f4405a6e21eadeeb0f58c83343de7ca1e0e1434e56bc1b" gracePeriod=15 Jul 18 11:50:59.063457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.063425 2578 generic.go:356] "Generic (PLEG): container finished" podID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" containerID="bda8af5265d028d3c1f4405a6e21eadeeb0f58c83343de7ca1e0e1434e56bc1b" exitCode=0 Jul 18 11:50:59.063635 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.063504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f85d96bc-8l4hf" event={"ID":"89c58ba2-c338-45b3-aec0-54e0b4d45c92","Type":"ContainerDied","Data":"bda8af5265d028d3c1f4405a6e21eadeeb0f58c83343de7ca1e0e1434e56bc1b"} Jul 18 11:50:59.094145 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.094122 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:50:59.271146 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271068 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271146 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271105 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271146 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271122 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271146 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271138 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271186 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271204 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271224 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2jf8\" (UniqueName: \"kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8\") pod \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\" (UID: \"89c58ba2-c338-45b3-aec0-54e0b4d45c92\") " Jul 18 11:50:59.271673 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271642 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:50:59.271793 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271691 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca" (OuterVolumeSpecName: "service-ca") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:50:59.271793 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271744 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config" (OuterVolumeSpecName: "console-config") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:50:59.271793 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.271743 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:50:59.273361 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.273339 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:50:59.273497 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.273477 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8" (OuterVolumeSpecName: "kube-api-access-p2jf8") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "kube-api-access-p2jf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:50:59.273539 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.273497 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "89c58ba2-c338-45b3-aec0-54e0b4d45c92" (UID: "89c58ba2-c338-45b3-aec0-54e0b4d45c92"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:50:59.372161 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372135 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-oauth-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372161 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372156 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372161 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372166 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-oauth-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372532 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372175 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-console-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372532 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372184 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372532 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372191 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89c58ba2-c338-45b3-aec0-54e0b4d45c92-trusted-ca-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:50:59.372532 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:50:59.372199 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2jf8\" (UniqueName: \"kubernetes.io/projected/89c58ba2-c338-45b3-aec0-54e0b4d45c92-kube-api-access-p2jf8\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.009652 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.009588 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerName="registry" containerID="cri-o://e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a" gracePeriod=30 Jul 18 11:51:00.068816 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.068783 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f85d96bc-8l4hf" Jul 18 11:51:00.068923 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.068784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f85d96bc-8l4hf" event={"ID":"89c58ba2-c338-45b3-aec0-54e0b4d45c92","Type":"ContainerDied","Data":"0a298950a4b7803942ae3b808e23280391c7585ccfb7ebbcef86be841f2bfb9a"} Jul 18 11:51:00.068982 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.068950 2578 scope.go:117] "RemoveContainer" containerID="bda8af5265d028d3c1f4405a6e21eadeeb0f58c83343de7ca1e0e1434e56bc1b" Jul 18 11:51:00.124784 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.124760 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:51:00.127282 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.127260 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f85d96bc-8l4hf"] Jul 18 11:51:00.180605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.180566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:51:00.183226 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.183203 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e785910e-4713-4292-acc8-d9d884f9d8db-metrics-certs\") pod \"network-metrics-daemon-wh7n8\" (UID: \"e785910e-4713-4292-acc8-d9d884f9d8db\") " pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:51:00.220307 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.220276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6q4p9\"" Jul 18 11:51:00.228457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.228431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wh7n8" Jul 18 11:51:00.282901 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.282877 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:51:00.382488 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382462 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9qtx\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382542 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382588 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382621 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382636 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382672 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.382855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382698 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets\") pod \"c2aa38c0-d758-49c1-ad20-3a614100e76c\" (UID: \"c2aa38c0-d758-49c1-ad20-3a614100e76c\") " Jul 18 11:51:00.383243 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.382925 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:00.383243 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.383018 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:00.385022 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.384994 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:51:00.385022 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.385002 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:51:00.385341 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.385316 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx" (OuterVolumeSpecName: "kube-api-access-d9qtx") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "kube-api-access-d9qtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:51:00.385467 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.385341 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:51:00.385467 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.385401 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:51:00.390842 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.390818 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wh7n8"] Jul 18 11:51:00.391711 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.391677 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c2aa38c0-d758-49c1-ad20-3a614100e76c" (UID: "c2aa38c0-d758-49c1-ad20-3a614100e76c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:51:00.393715 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:51:00.393690 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode785910e_4713_4292_acc8_d9d884f9d8db.slice/crio-4e13200ef5e2174530eb055bec563a80f92b64979c32c429d31bf93a5c90838e WatchSource:0}: Error finding container 4e13200ef5e2174530eb055bec563a80f92b64979c32c429d31bf93a5c90838e: Status 404 returned error can't find the container with id 4e13200ef5e2174530eb055bec563a80f92b64979c32c429d31bf93a5c90838e Jul 18 11:51:00.483740 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483707 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-installation-pull-secrets\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483740 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483735 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9qtx\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-kube-api-access-d9qtx\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483740 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483745 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-certificates\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483754 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2aa38c0-d758-49c1-ad20-3a614100e76c-trusted-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483764 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c2aa38c0-d758-49c1-ad20-3a614100e76c-image-registry-private-configuration\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483773 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-registry-tls\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483781 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c2aa38c0-d758-49c1-ad20-3a614100e76c-bound-sa-token\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:00.483967 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:00.483788 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c2aa38c0-d758-49c1-ad20-3a614100e76c-ca-trust-extracted\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:01.074590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.074562 2578 generic.go:356] "Generic (PLEG): container finished" podID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerID="e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a" exitCode=0 Jul 18 11:51:01.074709 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.074605 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" Jul 18 11:51:01.074709 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.074642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" event={"ID":"c2aa38c0-d758-49c1-ad20-3a614100e76c","Type":"ContainerDied","Data":"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a"} Jul 18 11:51:01.074709 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.074692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd96cb6c-kpns4" event={"ID":"c2aa38c0-d758-49c1-ad20-3a614100e76c","Type":"ContainerDied","Data":"6f0da406eac78127b8ba9542e338ff5c3bd0feb8924c972720066356539461ed"} Jul 18 11:51:01.074837 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.074712 2578 scope.go:117] "RemoveContainer" containerID="e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a" Jul 18 11:51:01.076816 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.076796 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh7n8" event={"ID":"e785910e-4713-4292-acc8-d9d884f9d8db","Type":"ContainerStarted","Data":"4e13200ef5e2174530eb055bec563a80f92b64979c32c429d31bf93a5c90838e"} Jul 18 11:51:01.083284 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.083217 2578 scope.go:117] "RemoveContainer" containerID="e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a" Jul 18 11:51:01.083548 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:51:01.083522 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a\": container with ID starting with e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a not found: ID does not exist" containerID="e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a" Jul 18 11:51:01.083651 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.083552 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a"} err="failed to get container status \"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a\": rpc error: code = NotFound desc = could not find container \"e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a\": container with ID starting with e7c37e76b65f84c21fe5f9e4fd38bf9930cbc8b3b8b8a9a5dc38b1b58ae2c95a not found: ID does not exist" Jul 18 11:51:01.095765 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.095744 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:51:01.099769 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.099743 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-cd96cb6c-kpns4"] Jul 18 11:51:01.411713 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.411642 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" path="/var/lib/kubelet/pods/89c58ba2-c338-45b3-aec0-54e0b4d45c92/volumes" Jul 18 11:51:01.412100 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:01.412085 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" path="/var/lib/kubelet/pods/c2aa38c0-d758-49c1-ad20-3a614100e76c/volumes" Jul 18 11:51:02.082893 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:02.082855 2578 generic.go:356] "Generic (PLEG): container finished" podID="52094835-811b-4c18-8436-f8e9d0aaf34e" containerID="caf7c137a3a108b74d225943755e9e6ccc13e4520a015514538233c2807a409e" exitCode=0 Jul 18 11:51:02.083072 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:02.082905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" event={"ID":"52094835-811b-4c18-8436-f8e9d0aaf34e","Type":"ContainerDied","Data":"caf7c137a3a108b74d225943755e9e6ccc13e4520a015514538233c2807a409e"} Jul 18 11:51:02.083247 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:02.083232 2578 scope.go:117] "RemoveContainer" containerID="caf7c137a3a108b74d225943755e9e6ccc13e4520a015514538233c2807a409e" Jul 18 11:51:03.092808 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:03.092761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh7n8" event={"ID":"e785910e-4713-4292-acc8-d9d884f9d8db","Type":"ContainerStarted","Data":"9dff8e763150b91a677aaabbf9cefe4bc99e1c4b71b66499b399cd13d2125f60"} Jul 18 11:51:03.093309 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:03.092877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wh7n8" event={"ID":"e785910e-4713-4292-acc8-d9d884f9d8db","Type":"ContainerStarted","Data":"9af44ac413ffcdc421d75ac149b04afe0cb286439a98e3eb76a5ed79065ddc39"} Jul 18 11:51:03.096507 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:03.096458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ndvr6" event={"ID":"52094835-811b-4c18-8436-f8e9d0aaf34e","Type":"ContainerStarted","Data":"7e9d7cac67a0081a4bc7fbfb91a9a00d5805ceee3a46a0d965a1147ad90d1e15"} Jul 18 11:51:03.107587 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:03.107545 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wh7n8" podStartSLOduration=130.403699068 podStartE2EDuration="2m12.107531374s" totalImagesPullingTime="1.703832306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:51:03.106603246 +0000 UTC m=+132.178662751" watchObservedRunningTime="2026-07-18 11:51:03.107531374 +0000 UTC m=+132.179590876" Jul 18 11:51:09.074242 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.074179 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5858dff758-f4lpv" podUID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" containerName="console" containerID="cri-o://59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59" gracePeriod=15 Jul 18 11:51:09.308353 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.308331 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:51:09.355269 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355203 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355410 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355273 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q7q8\" (UniqueName: \"kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355410 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355300 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355410 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355318 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355410 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355336 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355410 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355355 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert\") pod \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\" (UID: \"76d0a9f7-8e19-4b97-9545-56d7d94909f2\") " Jul 18 11:51:09.355655 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355619 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config" (OuterVolumeSpecName: "console-config") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:09.355711 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355655 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:09.355855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.355828 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:51:09.357503 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.357465 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:51:09.357608 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.357534 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:51:09.357608 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.357560 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8" (OuterVolumeSpecName: "kube-api-access-8q7q8") pod "76d0a9f7-8e19-4b97-9545-56d7d94909f2" (UID: "76d0a9f7-8e19-4b97-9545-56d7d94909f2"). InnerVolumeSpecName "kube-api-access-8q7q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:51:09.455972 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.455944 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q7q8\" (UniqueName: \"kubernetes.io/projected/76d0a9f7-8e19-4b97-9545-56d7d94909f2-kube-api-access-8q7q8\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:09.455972 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.455968 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:09.456153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.455977 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-oauth-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:09.456153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.455986 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:09.456153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.455994 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-oauth-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:09.456153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:09.456002 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76d0a9f7-8e19-4b97-9545-56d7d94909f2-console-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:51:10.124201 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.124163 2578 generic.go:356] "Generic (PLEG): container finished" podID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" containerID="59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59" exitCode=0 Jul 18 11:51:10.124657 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.124211 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5858dff758-f4lpv" Jul 18 11:51:10.124657 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.124252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858dff758-f4lpv" event={"ID":"76d0a9f7-8e19-4b97-9545-56d7d94909f2","Type":"ContainerDied","Data":"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59"} Jul 18 11:51:10.124657 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.124314 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5858dff758-f4lpv" event={"ID":"76d0a9f7-8e19-4b97-9545-56d7d94909f2","Type":"ContainerDied","Data":"b803fa1db0f92fdfaee934a53407e57745bbfc4ce35810fa4d7f3694c4b70d3c"} Jul 18 11:51:10.124657 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.124348 2578 scope.go:117] "RemoveContainer" containerID="59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59" Jul 18 11:51:10.132635 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.132608 2578 scope.go:117] "RemoveContainer" containerID="59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59" Jul 18 11:51:10.132883 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:51:10.132864 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59\": container with ID starting with 59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59 not found: ID does not exist" containerID="59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59" Jul 18 11:51:10.132931 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.132896 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59"} err="failed to get container status \"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59\": rpc error: code = NotFound desc = could not find container \"59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59\": container with ID starting with 59ffa80750bce2dd99c152c5b78fe733dfeb55b3d84603477a13416712e0ab59 not found: ID does not exist" Jul 18 11:51:10.140896 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.140871 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:51:10.144344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:10.144326 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5858dff758-f4lpv"] Jul 18 11:51:11.410649 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:11.410618 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" path="/var/lib/kubelet/pods/76d0a9f7-8e19-4b97-9545-56d7d94909f2/volumes" Jul 18 11:51:56.316250 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316217 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316515 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerName="registry" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316526 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerName="registry" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316550 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" containerName="console" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316572 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" containerName="console" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316582 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" containerName="console" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316587 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" containerName="console" Jul 18 11:51:56.316642 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316638 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d0a9f7-8e19-4b97-9545-56d7d94909f2" containerName="console" Jul 18 11:51:56.316853 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316647 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2aa38c0-d758-49c1-ad20-3a614100e76c" containerName="registry" Jul 18 11:51:56.316853 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.316653 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c58ba2-c338-45b3-aec0-54e0b4d45c92" containerName="console" Jul 18 11:51:56.319404 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.319385 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.331748 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.331725 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:51:56.411679 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-984cx\" (UniqueName: \"kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.411838 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.411782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512712 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512840 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-984cx\" (UniqueName: \"kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.512906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.512884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.513499 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.513475 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.513636 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.513615 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.513840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.513817 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.514095 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.514075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.515295 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.515276 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.515422 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.515403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.521263 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.521237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-984cx\" (UniqueName: \"kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx\") pod \"console-6cd577d4f9-ksl9h\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.628188 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.628093 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:51:56.774521 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:56.774457 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:51:56.777651 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:51:56.777624 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08003959_d811_4ef6_bc42_ba19cc86b82f.slice/crio-f47bac6ad25ad384b5f651f9d4c8cea16cb688fdcbe6029e7312a7b40464a06c WatchSource:0}: Error finding container f47bac6ad25ad384b5f651f9d4c8cea16cb688fdcbe6029e7312a7b40464a06c: Status 404 returned error can't find the container with id f47bac6ad25ad384b5f651f9d4c8cea16cb688fdcbe6029e7312a7b40464a06c Jul 18 11:51:57.318606 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:57.318569 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd577d4f9-ksl9h" event={"ID":"08003959-d811-4ef6-bc42-ba19cc86b82f","Type":"ContainerStarted","Data":"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f"} Jul 18 11:51:57.318959 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:57.318617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd577d4f9-ksl9h" event={"ID":"08003959-d811-4ef6-bc42-ba19cc86b82f","Type":"ContainerStarted","Data":"f47bac6ad25ad384b5f651f9d4c8cea16cb688fdcbe6029e7312a7b40464a06c"} Jul 18 11:51:57.339364 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:51:57.339313 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd577d4f9-ksl9h" podStartSLOduration=1.3393005119999999 podStartE2EDuration="1.339300512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:51:57.338231387 +0000 UTC m=+186.410290891" watchObservedRunningTime="2026-07-18 11:51:57.339300512 +0000 UTC m=+186.411360015" Jul 18 11:52:06.390703 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.390670 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:52:06.423800 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.423767 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:52:06.426175 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.426159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.436532 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.436511 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:52:06.600328 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600328 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600326 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600541 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600359 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600541 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600541 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhk9c\" (UniqueName: \"kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600541 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.600663 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.600545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.628338 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.628312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:52:06.701975 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.701942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.701990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.702017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.702068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.702095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.702118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhk9c\" (UniqueName: \"kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.702729 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.702179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.703106 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.703019 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.703106 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.703087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.703205 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.703149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.703205 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.703182 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.704751 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.704731 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.704884 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.704860 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.709712 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.709688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhk9c\" (UniqueName: \"kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c\") pod \"console-c9bcd7d47-nn565\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.735780 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.735746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:06.878287 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:06.878264 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:52:06.880945 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:52:06.880915 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cb69cc_3983_428e_96a7_0cefda5e3dbf.slice/crio-a965b1779f62e765a54276776a236a35ac4a46bd474e95d7617146c658d61770 WatchSource:0}: Error finding container a965b1779f62e765a54276776a236a35ac4a46bd474e95d7617146c658d61770: Status 404 returned error can't find the container with id a965b1779f62e765a54276776a236a35ac4a46bd474e95d7617146c658d61770 Jul 18 11:52:07.357586 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:07.357552 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bcd7d47-nn565" event={"ID":"48cb69cc-3983-428e-96a7-0cefda5e3dbf","Type":"ContainerStarted","Data":"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b"} Jul 18 11:52:07.357750 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:07.357602 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bcd7d47-nn565" event={"ID":"48cb69cc-3983-428e-96a7-0cefda5e3dbf","Type":"ContainerStarted","Data":"a965b1779f62e765a54276776a236a35ac4a46bd474e95d7617146c658d61770"} Jul 18 11:52:07.373264 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:07.373217 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9bcd7d47-nn565" podStartSLOduration=1.373204382 podStartE2EDuration="1.373204382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:52:07.372217327 +0000 UTC m=+196.444276829" watchObservedRunningTime="2026-07-18 11:52:07.373204382 +0000 UTC m=+196.445263951" Jul 18 11:52:16.736540 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:16.736499 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:16.736945 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:16.736566 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:16.742453 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:16.742428 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:17.403799 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:17.403769 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:52:17.449426 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:17.449392 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:52:31.410140 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.410081 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cd577d4f9-ksl9h" podUID="08003959-d811-4ef6-bc42-ba19cc86b82f" containerName="console" containerID="cri-o://340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f" gracePeriod=15 Jul 18 11:52:31.652562 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.652540 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:52:31.695293 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695267 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695433 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695433 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695316 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695433 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695433 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695406 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695645 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695485 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-984cx\" (UniqueName: \"kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695645 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695523 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca\") pod \"08003959-d811-4ef6-bc42-ba19cc86b82f\" (UID: \"08003959-d811-4ef6-bc42-ba19cc86b82f\") " Jul 18 11:52:31.695734 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695715 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config" (OuterVolumeSpecName: "console-config") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:31.695785 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695729 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:31.695840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695813 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-oauth-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.695840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695814 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:31.695840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.695832 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-console-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.696069 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.696030 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca" (OuterVolumeSpecName: "service-ca") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:31.697476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.697447 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:52:31.697476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.697460 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:52:31.697600 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.697536 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx" (OuterVolumeSpecName: "kube-api-access-984cx") pod "08003959-d811-4ef6-bc42-ba19cc86b82f" (UID: "08003959-d811-4ef6-bc42-ba19cc86b82f"). InnerVolumeSpecName "kube-api-access-984cx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:52:31.796786 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.796757 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-oauth-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.796786 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.796781 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08003959-d811-4ef6-bc42-ba19cc86b82f-console-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.796786 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.796791 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-trusted-ca-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.796975 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.796800 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-984cx\" (UniqueName: \"kubernetes.io/projected/08003959-d811-4ef6-bc42-ba19cc86b82f-kube-api-access-984cx\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:31.796975 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:31.796811 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08003959-d811-4ef6-bc42-ba19cc86b82f-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:32.456598 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.456563 2578 generic.go:356] "Generic (PLEG): container finished" podID="08003959-d811-4ef6-bc42-ba19cc86b82f" containerID="340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f" exitCode=0 Jul 18 11:52:32.456598 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.456600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd577d4f9-ksl9h" event={"ID":"08003959-d811-4ef6-bc42-ba19cc86b82f","Type":"ContainerDied","Data":"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f"} Jul 18 11:52:32.457086 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.456633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd577d4f9-ksl9h" event={"ID":"08003959-d811-4ef6-bc42-ba19cc86b82f","Type":"ContainerDied","Data":"f47bac6ad25ad384b5f651f9d4c8cea16cb688fdcbe6029e7312a7b40464a06c"} Jul 18 11:52:32.457086 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.456651 2578 scope.go:117] "RemoveContainer" containerID="340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f" Jul 18 11:52:32.457086 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.456603 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd577d4f9-ksl9h" Jul 18 11:52:32.465382 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.465363 2578 scope.go:117] "RemoveContainer" containerID="340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f" Jul 18 11:52:32.465621 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:52:32.465605 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f\": container with ID starting with 340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f not found: ID does not exist" containerID="340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f" Jul 18 11:52:32.465672 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.465628 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f"} err="failed to get container status \"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f\": rpc error: code = NotFound desc = could not find container \"340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f\": container with ID starting with 340162dc2563e3c6c819aa4114d9e60a58178e9e12e6d6f889e99e47a77aeb7f not found: ID does not exist" Jul 18 11:52:32.480113 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.480087 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:52:32.484430 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:32.484411 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd577d4f9-ksl9h"] Jul 18 11:52:33.410952 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:33.410920 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08003959-d811-4ef6-bc42-ba19cc86b82f" path="/var/lib/kubelet/pods/08003959-d811-4ef6-bc42-ba19cc86b82f/volumes" Jul 18 11:52:42.469488 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.469440 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66f8c65c4-tsg7s" podUID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" containerName="console" containerID="cri-o://c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f" gracePeriod=15 Jul 18 11:52:42.702378 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.702356 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:52:42.779289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779203 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779288 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nql46\" (UniqueName: \"kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779323 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779345 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779367 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779402 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779439 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779427 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca\") pod \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\" (UID: \"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0\") " Jul 18 11:52:42.779726 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779692 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config" (OuterVolumeSpecName: "console-config") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:42.779726 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779698 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:42.779831 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779732 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:42.779954 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.779934 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca" (OuterVolumeSpecName: "service-ca") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:52:42.781518 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.781491 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46" (OuterVolumeSpecName: "kube-api-access-nql46") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "kube-api-access-nql46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:52:42.781518 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.781500 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:52:42.781641 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.781516 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" (UID: "45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:52:42.880599 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880568 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880599 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880598 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-oauth-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880599 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880608 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-oauth-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880818 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880617 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-console-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880818 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880625 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880818 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880633 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-trusted-ca-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:42.880818 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:42.880641 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nql46\" (UniqueName: \"kubernetes.io/projected/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0-kube-api-access-nql46\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:52:43.501767 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.501737 2578 generic.go:356] "Generic (PLEG): container finished" podID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" containerID="c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f" exitCode=0 Jul 18 11:52:43.502172 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.501783 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f8c65c4-tsg7s" Jul 18 11:52:43.502172 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.501820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f8c65c4-tsg7s" event={"ID":"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0","Type":"ContainerDied","Data":"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f"} Jul 18 11:52:43.502172 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.501868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f8c65c4-tsg7s" event={"ID":"45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0","Type":"ContainerDied","Data":"05c927dc7e8a4fa8d1d89fecd9b1c677c064b13b869d1ee04af088ebc1c937ef"} Jul 18 11:52:43.502172 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.501887 2578 scope.go:117] "RemoveContainer" containerID="c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f" Jul 18 11:52:43.509792 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.509777 2578 scope.go:117] "RemoveContainer" containerID="c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f" Jul 18 11:52:43.510039 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:52:43.510023 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f\": container with ID starting with c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f not found: ID does not exist" containerID="c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f" Jul 18 11:52:43.510099 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.510066 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f"} err="failed to get container status \"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f\": rpc error: code = NotFound desc = could not find container \"c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f\": container with ID starting with c42b8eebefc9d701a29b4c0c33222711245711ef8831538f39ee11a0c42c985f not found: ID does not exist" Jul 18 11:52:43.517438 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.517410 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:52:43.518662 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:43.518643 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66f8c65c4-tsg7s"] Jul 18 11:52:45.410853 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:45.410819 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" path="/var/lib/kubelet/pods/45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0/volumes" Jul 18 11:52:50.234170 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234133 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw"] Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234417 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08003959-d811-4ef6-bc42-ba19cc86b82f" containerName="console" Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234429 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="08003959-d811-4ef6-bc42-ba19cc86b82f" containerName="console" Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234444 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" containerName="console" Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234450 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" containerName="console" Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234499 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="08003959-d811-4ef6-bc42-ba19cc86b82f" containerName="console" Jul 18 11:52:50.234526 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.234509 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e9ad3a-2e7f-4289-8df2-5e5c3f4866b0" containerName="console" Jul 18 11:52:50.237229 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.237212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.239650 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.239626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:52:50.240389 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.240362 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:52:50.240506 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.240362 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xlvk6\"" Jul 18 11:52:50.245288 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.245266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw"] Jul 18 11:52:50.334878 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.334834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.334878 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.334877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.335121 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.334955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4wz\" (UniqueName: \"kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.435426 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.435389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4wz\" (UniqueName: \"kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.435610 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.435464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.435610 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.435483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.435855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.435833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.435922 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.435882 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.443237 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.443208 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4wz\" (UniqueName: \"kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.546712 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.546623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:52:50.669682 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:50.669649 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw"] Jul 18 11:52:50.672514 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:52:50.672488 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fa651a_2c48_45d0_9c22_21a95c105485.slice/crio-b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7 WatchSource:0}: Error finding container b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7: Status 404 returned error can't find the container with id b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7 Jul 18 11:52:51.533564 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:51.533518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerStarted","Data":"b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7"} Jul 18 11:52:56.558365 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:56.558334 2578 generic.go:356] "Generic (PLEG): container finished" podID="71fa651a-2c48-45d0-9c22-21a95c105485" containerID="e6ffc1485bccd1d30558de35a908500014a401c34cc2971c2e6dddb1c6cf2ac2" exitCode=0 Jul 18 11:52:56.558714 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:56.558395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerDied","Data":"e6ffc1485bccd1d30558de35a908500014a401c34cc2971c2e6dddb1c6cf2ac2"} Jul 18 11:52:58.568127 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:58.568098 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerStarted","Data":"e19ea52b4870cd0a7c82f2e88c2ad80b9bf9c543be92d6fba97aa2fbfcd30c2a"} Jul 18 11:52:59.573684 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:59.573650 2578 generic.go:356] "Generic (PLEG): container finished" podID="71fa651a-2c48-45d0-9c22-21a95c105485" containerID="e19ea52b4870cd0a7c82f2e88c2ad80b9bf9c543be92d6fba97aa2fbfcd30c2a" exitCode=0 Jul 18 11:52:59.574039 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:52:59.573742 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerDied","Data":"e19ea52b4870cd0a7c82f2e88c2ad80b9bf9c543be92d6fba97aa2fbfcd30c2a"} Jul 18 11:53:06.605581 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:06.605548 2578 generic.go:356] "Generic (PLEG): container finished" podID="71fa651a-2c48-45d0-9c22-21a95c105485" containerID="1b60ba6b0adc4d3f31e651f3f954a4a78349a4e18f3462a58ccb685c224ae896" exitCode=0 Jul 18 11:53:06.605976 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:06.605633 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerDied","Data":"1b60ba6b0adc4d3f31e651f3f954a4a78349a4e18f3462a58ccb685c224ae896"} Jul 18 11:53:07.731395 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.731373 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:53:07.895303 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.895223 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb4wz\" (UniqueName: \"kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz\") pod \"71fa651a-2c48-45d0-9c22-21a95c105485\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " Jul 18 11:53:07.895437 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.895320 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util\") pod \"71fa651a-2c48-45d0-9c22-21a95c105485\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " Jul 18 11:53:07.895437 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.895355 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle\") pod \"71fa651a-2c48-45d0-9c22-21a95c105485\" (UID: \"71fa651a-2c48-45d0-9c22-21a95c105485\") " Jul 18 11:53:07.895986 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.895952 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle" (OuterVolumeSpecName: "bundle") pod "71fa651a-2c48-45d0-9c22-21a95c105485" (UID: "71fa651a-2c48-45d0-9c22-21a95c105485"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:07.897372 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.897346 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz" (OuterVolumeSpecName: "kube-api-access-fb4wz") pod "71fa651a-2c48-45d0-9c22-21a95c105485" (UID: "71fa651a-2c48-45d0-9c22-21a95c105485"). InnerVolumeSpecName "kube-api-access-fb4wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:53:07.900712 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.900687 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util" (OuterVolumeSpecName: "util") pod "71fa651a-2c48-45d0-9c22-21a95c105485" (UID: "71fa651a-2c48-45d0-9c22-21a95c105485"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:07.995861 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.995816 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:07.995861 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.995859 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb4wz\" (UniqueName: \"kubernetes.io/projected/71fa651a-2c48-45d0-9c22-21a95c105485-kube-api-access-fb4wz\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:07.995861 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:07.995869 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71fa651a-2c48-45d0-9c22-21a95c105485-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:08.614235 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:08.614193 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" Jul 18 11:53:08.614235 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:08.614185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jj6fw" event={"ID":"71fa651a-2c48-45d0-9c22-21a95c105485","Type":"ContainerDied","Data":"b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7"} Jul 18 11:53:08.614479 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:08.614295 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0a5cf364a6813fcc6b60af7564a76c159d2b9254cb77639fc63bbc8a0305db7" Jul 18 11:53:13.215691 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215652 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc"] Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215926 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="pull" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215936 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="pull" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215954 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="util" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215959 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="util" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215969 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="extract" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.215975 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="extract" Jul 18 11:53:13.216110 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.216024 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="71fa651a-2c48-45d0-9c22-21a95c105485" containerName="extract" Jul 18 11:53:13.218577 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.218561 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.220931 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.220912 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 18 11:53:13.220931 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.220934 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-plxwz\"" Jul 18 11:53:13.221279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.221263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:53:13.232514 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.232495 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc"] Jul 18 11:53:13.339683 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.339643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpddx\" (UniqueName: \"kubernetes.io/projected/aa917ab6-1683-4814-ba77-8d4f5b348a03-kube-api-access-mpddx\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.339858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.339690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa917ab6-1683-4814-ba77-8d4f5b348a03-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.440996 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.440966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpddx\" (UniqueName: \"kubernetes.io/projected/aa917ab6-1683-4814-ba77-8d4f5b348a03-kube-api-access-mpddx\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.440996 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.441000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa917ab6-1683-4814-ba77-8d4f5b348a03-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.441429 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.441409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa917ab6-1683-4814-ba77-8d4f5b348a03-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.450292 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.450267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpddx\" (UniqueName: \"kubernetes.io/projected/aa917ab6-1683-4814-ba77-8d4f5b348a03-kube-api-access-mpddx\") pod \"cert-manager-operator-controller-manager-55c444bf86-x8fdc\" (UID: \"aa917ab6-1683-4814-ba77-8d4f5b348a03\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.527931 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.527854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" Jul 18 11:53:13.657616 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:13.657587 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc"] Jul 18 11:53:13.660606 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:13.660582 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa917ab6_1683_4814_ba77_8d4f5b348a03.slice/crio-9a842029545b3de63553c6c54c23259ab22dd02c65753de6635c68deb88e626d WatchSource:0}: Error finding container 9a842029545b3de63553c6c54c23259ab22dd02c65753de6635c68deb88e626d: Status 404 returned error can't find the container with id 9a842029545b3de63553c6c54c23259ab22dd02c65753de6635c68deb88e626d Jul 18 11:53:14.638405 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:14.638368 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" event={"ID":"aa917ab6-1683-4814-ba77-8d4f5b348a03","Type":"ContainerStarted","Data":"9a842029545b3de63553c6c54c23259ab22dd02c65753de6635c68deb88e626d"} Jul 18 11:53:15.649041 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:15.648958 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" event={"ID":"aa917ab6-1683-4814-ba77-8d4f5b348a03","Type":"ContainerStarted","Data":"a7d5ef0e825fcebb22430f887589312920df64546e648a5ad8006d51f3a08d8b"} Jul 18 11:53:15.670037 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:15.669986 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x8fdc" podStartSLOduration=1.158526012 podStartE2EDuration="2.669970297s" totalImagesPullingTime="1.511444285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:53:15.667864058 +0000 UTC m=+264.739923561" watchObservedRunningTime="2026-07-18 11:53:15.669970297 +0000 UTC m=+264.742029799" Jul 18 11:53:16.707474 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.707438 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ts5mf"] Jul 18 11:53:16.710649 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.710632 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.712894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.712870 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 18 11:53:16.713003 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.712893 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 18 11:53:16.713515 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.713499 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-d2cns\"" Jul 18 11:53:16.717816 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.717566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ts5mf"] Jul 18 11:53:16.868911 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.868878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.869115 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.868930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwtlw\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-kube-api-access-pwtlw\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.970257 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.970179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwtlw\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-kube-api-access-pwtlw\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.970257 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.970248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.978323 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.978295 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwtlw\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-kube-api-access-pwtlw\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:16.978323 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:16.978300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77026b8a-0560-403c-ac23-828cfc44d8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ts5mf\" (UID: \"77026b8a-0560-403c-ac23-828cfc44d8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:17.031249 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.031220 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:17.158692 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.158668 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ts5mf"] Jul 18 11:53:17.160926 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:17.160894 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77026b8a_0560_403c_ac23_828cfc44d8f5.slice/crio-92b3e6d0269b13bb4f46db50e0a6ee35a7c9557d5a50c3227ca34987f3cf6045 WatchSource:0}: Error finding container 92b3e6d0269b13bb4f46db50e0a6ee35a7c9557d5a50c3227ca34987f3cf6045: Status 404 returned error can't find the container with id 92b3e6d0269b13bb4f46db50e0a6ee35a7c9557d5a50c3227ca34987f3cf6045 Jul 18 11:53:17.534353 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.534320 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n"] Jul 18 11:53:17.538968 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.538951 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.541270 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.541245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:53:17.542118 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.542100 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:53:17.542118 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.542111 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xlvk6\"" Jul 18 11:53:17.545850 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.545829 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n"] Jul 18 11:53:17.659157 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.659111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" event={"ID":"77026b8a-0560-403c-ac23-828cfc44d8f5","Type":"ContainerStarted","Data":"92b3e6d0269b13bb4f46db50e0a6ee35a7c9557d5a50c3227ca34987f3cf6045"} Jul 18 11:53:17.676545 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.676511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.676690 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.676599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh7rr\" (UniqueName: \"kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.676690 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.676659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.777457 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.777421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.777938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.777507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.777938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.777554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh7rr\" (UniqueName: \"kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.777938 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.777927 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.778088 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.777978 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.785858 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.785807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh7rr\" (UniqueName: \"kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.849651 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.849620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:17.996248 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:17.996219 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n"] Jul 18 11:53:17.999157 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:17.999119 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd085c42e_fc43_4656_9bae_64c57f83659d.slice/crio-4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff WatchSource:0}: Error finding container 4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff: Status 404 returned error can't find the container with id 4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff Jul 18 11:53:18.665106 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:18.665070 2578 generic.go:356] "Generic (PLEG): container finished" podID="d085c42e-fc43-4656-9bae-64c57f83659d" containerID="14e949cfdfa5c538fe761ceb367bdcc3893224785ee33a3c4e706dde406aadfa" exitCode=0 Jul 18 11:53:18.665282 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:18.665109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" event={"ID":"d085c42e-fc43-4656-9bae-64c57f83659d","Type":"ContainerDied","Data":"14e949cfdfa5c538fe761ceb367bdcc3893224785ee33a3c4e706dde406aadfa"} Jul 18 11:53:18.665282 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:18.665166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" event={"ID":"d085c42e-fc43-4656-9bae-64c57f83659d","Type":"ContainerStarted","Data":"4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff"} Jul 18 11:53:19.670381 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:19.670354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" event={"ID":"77026b8a-0560-403c-ac23-828cfc44d8f5","Type":"ContainerStarted","Data":"8f24d80f220bf9f1f302d0ed58964d3f826ccafc6713cd81353763cf2bd1e830"} Jul 18 11:53:19.670795 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:19.670480 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:19.687816 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:19.687753 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" podStartSLOduration=1.258809315 podStartE2EDuration="3.687736409s" totalImagesPullingTime="2.428927094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:53:19.686365749 +0000 UTC m=+268.758425255" watchObservedRunningTime="2026-07-18 11:53:19.687736409 +0000 UTC m=+268.759795912" Jul 18 11:53:21.680707 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:21.680673 2578 generic.go:356] "Generic (PLEG): container finished" podID="d085c42e-fc43-4656-9bae-64c57f83659d" containerID="9598ae2115c719681a51392413f0e645cd1fb4fd6e0a6a9d1fe17a863657ce10" exitCode=0 Jul 18 11:53:21.681065 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:21.680728 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" event={"ID":"d085c42e-fc43-4656-9bae-64c57f83659d","Type":"ContainerDied","Data":"9598ae2115c719681a51392413f0e645cd1fb4fd6e0a6a9d1fe17a863657ce10"} Jul 18 11:53:22.687813 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:22.687775 2578 generic.go:356] "Generic (PLEG): container finished" podID="d085c42e-fc43-4656-9bae-64c57f83659d" containerID="2038dd5bb6e57d43fc02cbe078d47df4c1e4f42e96772568820ffa53e2c406eb" exitCode=0 Jul 18 11:53:22.688272 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:22.687866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" event={"ID":"d085c42e-fc43-4656-9bae-64c57f83659d","Type":"ContainerDied","Data":"2038dd5bb6e57d43fc02cbe078d47df4c1e4f42e96772568820ffa53e2c406eb"} Jul 18 11:53:23.820471 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.820447 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:23.927652 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.927623 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle\") pod \"d085c42e-fc43-4656-9bae-64c57f83659d\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " Jul 18 11:53:23.927826 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.927665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util\") pod \"d085c42e-fc43-4656-9bae-64c57f83659d\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " Jul 18 11:53:23.927826 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.927723 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh7rr\" (UniqueName: \"kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr\") pod \"d085c42e-fc43-4656-9bae-64c57f83659d\" (UID: \"d085c42e-fc43-4656-9bae-64c57f83659d\") " Jul 18 11:53:23.928104 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.928074 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle" (OuterVolumeSpecName: "bundle") pod "d085c42e-fc43-4656-9bae-64c57f83659d" (UID: "d085c42e-fc43-4656-9bae-64c57f83659d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:23.929786 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.929753 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr" (OuterVolumeSpecName: "kube-api-access-jh7rr") pod "d085c42e-fc43-4656-9bae-64c57f83659d" (UID: "d085c42e-fc43-4656-9bae-64c57f83659d"). InnerVolumeSpecName "kube-api-access-jh7rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:53:23.932240 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:23.932214 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util" (OuterVolumeSpecName: "util") pod "d085c42e-fc43-4656-9bae-64c57f83659d" (UID: "d085c42e-fc43-4656-9bae-64c57f83659d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:24.029071 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.028978 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:24.029071 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.029004 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d085c42e-fc43-4656-9bae-64c57f83659d-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:24.029071 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.029013 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh7rr\" (UniqueName: \"kubernetes.io/projected/d085c42e-fc43-4656-9bae-64c57f83659d-kube-api-access-jh7rr\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:24.698321 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.698280 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" event={"ID":"d085c42e-fc43-4656-9bae-64c57f83659d","Type":"ContainerDied","Data":"4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff"} Jul 18 11:53:24.698321 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.698326 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fg5r2n" Jul 18 11:53:24.698547 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:24.698340 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d72ca9312bf46b272e4e34d834851c45978b1dc859200b3aefeb87dd6f011ff" Jul 18 11:53:25.678076 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:25.678025 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ts5mf" Jul 18 11:53:26.091295 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091218 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fhdxj"] Jul 18 11:53:26.091558 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091542 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="util" Jul 18 11:53:26.091605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091562 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="util" Jul 18 11:53:26.091605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091573 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="extract" Jul 18 11:53:26.091605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091579 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="extract" Jul 18 11:53:26.091605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091586 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="pull" Jul 18 11:53:26.091605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091591 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="pull" Jul 18 11:53:26.091766 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.091660 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d085c42e-fc43-4656-9bae-64c57f83659d" containerName="extract" Jul 18 11:53:26.095483 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.095466 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.097654 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.097636 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rjrng\"" Jul 18 11:53:26.104234 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.104210 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fhdxj"] Jul 18 11:53:26.142766 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.142736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-bound-sa-token\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.142907 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.142786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb4xd\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-kube-api-access-lb4xd\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.243865 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.243833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb4xd\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-kube-api-access-lb4xd\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.244009 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.243889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-bound-sa-token\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.251552 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.251524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-bound-sa-token\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.251648 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.251557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb4xd\" (UniqueName: \"kubernetes.io/projected/965a8cbc-9865-480d-a1ae-f2a91243dcac-kube-api-access-lb4xd\") pod \"cert-manager-55b4d954d5-fhdxj\" (UID: \"965a8cbc-9865-480d-a1ae-f2a91243dcac\") " pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.404325 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.404241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fhdxj" Jul 18 11:53:26.543723 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.543699 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fhdxj"] Jul 18 11:53:26.546193 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:26.546160 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod965a8cbc_9865_480d_a1ae_f2a91243dcac.slice/crio-3c11aa71b25166fccf2320495edc683f91a96e44b78f474fb53bc80d1a68335d WatchSource:0}: Error finding container 3c11aa71b25166fccf2320495edc683f91a96e44b78f474fb53bc80d1a68335d: Status 404 returned error can't find the container with id 3c11aa71b25166fccf2320495edc683f91a96e44b78f474fb53bc80d1a68335d Jul 18 11:53:26.709890 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.709858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fhdxj" event={"ID":"965a8cbc-9865-480d-a1ae-f2a91243dcac","Type":"ContainerStarted","Data":"98c8e9136a1da2be8ac0de63ddf1021e1cc8972a7788e09f20c757e3c768ce79"} Jul 18 11:53:26.710276 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.709899 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fhdxj" event={"ID":"965a8cbc-9865-480d-a1ae-f2a91243dcac","Type":"ContainerStarted","Data":"3c11aa71b25166fccf2320495edc683f91a96e44b78f474fb53bc80d1a68335d"} Jul 18 11:53:26.724506 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:26.724461 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fhdxj" podStartSLOduration=0.724446708 podStartE2EDuration="724.446708ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:53:26.724290339 +0000 UTC m=+275.796349845" watchObservedRunningTime="2026-07-18 11:53:26.724446708 +0000 UTC m=+275.796506210" Jul 18 11:53:36.223220 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.223184 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2"] Jul 18 11:53:36.263422 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.263384 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2"] Jul 18 11:53:36.263577 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.263517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.266060 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.266023 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 11:53:36.266180 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.266025 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xlvk6\"" Jul 18 11:53:36.266898 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.266874 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 11:53:36.316252 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.316209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.316426 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.316271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvhw8\" (UniqueName: \"kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.316426 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.316353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.417073 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.417012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvhw8\" (UniqueName: \"kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.417073 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.417073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.417359 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.417104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.417493 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.417476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.417538 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.417512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.424836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.424807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvhw8\" (UniqueName: \"kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.573803 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.573723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:36.715836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.715804 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2"] Jul 18 11:53:36.719163 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:36.719136 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f10255b_5409_4b0b_849f_bfb3e4c81798.slice/crio-3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d WatchSource:0}: Error finding container 3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d: Status 404 returned error can't find the container with id 3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d Jul 18 11:53:36.756840 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:36.756811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" event={"ID":"8f10255b-5409-4b0b-849f-bfb3e4c81798","Type":"ContainerStarted","Data":"3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d"} Jul 18 11:53:37.762579 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:37.762496 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerID="271e348243662a1a32b732b8ebd130f5517d50dc9f6d666339a21d841faa91c3" exitCode=0 Jul 18 11:53:37.762579 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:37.762548 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" event={"ID":"8f10255b-5409-4b0b-849f-bfb3e4c81798","Type":"ContainerDied","Data":"271e348243662a1a32b732b8ebd130f5517d50dc9f6d666339a21d841faa91c3"} Jul 18 11:53:39.772871 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:39.772838 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerID="a587ebeb2cbe3ae22dd0939a48a9595f87a1ca163ff6dffe2c42e87d4791c59e" exitCode=0 Jul 18 11:53:39.773339 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:39.772915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" event={"ID":"8f10255b-5409-4b0b-849f-bfb3e4c81798","Type":"ContainerDied","Data":"a587ebeb2cbe3ae22dd0939a48a9595f87a1ca163ff6dffe2c42e87d4791c59e"} Jul 18 11:53:40.779364 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:40.779329 2578 generic.go:356] "Generic (PLEG): container finished" podID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerID="4c34a1f53662052ed38bdb50572d9547ed319d830cb5fda929de30e6756ee985" exitCode=0 Jul 18 11:53:40.779745 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:40.779411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" event={"ID":"8f10255b-5409-4b0b-849f-bfb3e4c81798","Type":"ContainerDied","Data":"4c34a1f53662052ed38bdb50572d9547ed319d830cb5fda929de30e6756ee985"} Jul 18 11:53:41.905086 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.905037 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:41.962820 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.962785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle\") pod \"8f10255b-5409-4b0b-849f-bfb3e4c81798\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " Jul 18 11:53:41.962820 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.962825 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util\") pod \"8f10255b-5409-4b0b-849f-bfb3e4c81798\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " Jul 18 11:53:41.963069 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.962856 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvhw8\" (UniqueName: \"kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8\") pod \"8f10255b-5409-4b0b-849f-bfb3e4c81798\" (UID: \"8f10255b-5409-4b0b-849f-bfb3e4c81798\") " Jul 18 11:53:41.963476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.963453 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle" (OuterVolumeSpecName: "bundle") pod "8f10255b-5409-4b0b-849f-bfb3e4c81798" (UID: "8f10255b-5409-4b0b-849f-bfb3e4c81798"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:41.964888 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.964866 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8" (OuterVolumeSpecName: "kube-api-access-kvhw8") pod "8f10255b-5409-4b0b-849f-bfb3e4c81798" (UID: "8f10255b-5409-4b0b-849f-bfb3e4c81798"). InnerVolumeSpecName "kube-api-access-kvhw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:53:41.975016 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:41.974991 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util" (OuterVolumeSpecName: "util") pod "8f10255b-5409-4b0b-849f-bfb3e4c81798" (UID: "8f10255b-5409-4b0b-849f-bfb3e4c81798"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:53:42.064095 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.063980 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:42.064095 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.064008 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f10255b-5409-4b0b-849f-bfb3e4c81798-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:42.064095 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.064016 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvhw8\" (UniqueName: \"kubernetes.io/projected/8f10255b-5409-4b0b-849f-bfb3e4c81798-kube-api-access-kvhw8\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:53:42.789509 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.789474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" event={"ID":"8f10255b-5409-4b0b-849f-bfb3e4c81798","Type":"ContainerDied","Data":"3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d"} Jul 18 11:53:42.789677 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.789515 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvbxk2" Jul 18 11:53:42.789677 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:42.789522 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b04e217fd1aebb262bdee0f21c764635c11c26b3d89e92086fa78e9a807963d" Jul 18 11:53:46.685774 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.685744 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x"] Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686059 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="extract" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686071 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="extract" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686083 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="util" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686088 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="util" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686097 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="pull" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686102 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="pull" Jul 18 11:53:46.686289 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.686148 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f10255b-5409-4b0b-849f-bfb3e4c81798" containerName="extract" Jul 18 11:53:46.688811 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.688794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.691897 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.691874 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 11:53:46.692027 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.691912 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 11:53:46.692027 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.692001 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 11:53:46.692027 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.692001 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tgpq8\"" Jul 18 11:53:46.692200 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.692019 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 11:53:46.692200 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.692088 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 11:53:46.706447 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.706424 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x"] Jul 18 11:53:46.798833 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.798798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.799003 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.798897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glvxn\" (UniqueName: \"kubernetes.io/projected/110995e9-ae10-4681-8274-2d98ca9652fb-kube-api-access-glvxn\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.799003 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.798940 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/110995e9-ae10-4681-8274-2d98ca9652fb-manager-config\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.799003 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.798962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-metrics-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.900091 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.900038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glvxn\" (UniqueName: \"kubernetes.io/projected/110995e9-ae10-4681-8274-2d98ca9652fb-kube-api-access-glvxn\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.900258 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.900098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/110995e9-ae10-4681-8274-2d98ca9652fb-manager-config\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.900258 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.900136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-metrics-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.900332 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.900280 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.900824 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.900804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/110995e9-ae10-4681-8274-2d98ca9652fb-manager-config\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.902652 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.902628 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.902743 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.902706 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/110995e9-ae10-4681-8274-2d98ca9652fb-metrics-cert\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.912296 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.912273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glvxn\" (UniqueName: \"kubernetes.io/projected/110995e9-ae10-4681-8274-2d98ca9652fb-kube-api-access-glvxn\") pod \"lws-controller-manager-77c8bd676b-xg55x\" (UID: \"110995e9-ae10-4681-8274-2d98ca9652fb\") " pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:46.998195 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:46.998114 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:47.124821 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:47.124797 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x"] Jul 18 11:53:47.126624 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:47.126600 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110995e9_ae10_4681_8274_2d98ca9652fb.slice/crio-d45f5e319c3dceb4e436113e547d38de96be917544dcbcff81ca404ff147b315 WatchSource:0}: Error finding container d45f5e319c3dceb4e436113e547d38de96be917544dcbcff81ca404ff147b315: Status 404 returned error can't find the container with id d45f5e319c3dceb4e436113e547d38de96be917544dcbcff81ca404ff147b315 Jul 18 11:53:47.812997 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:47.812967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" event={"ID":"110995e9-ae10-4681-8274-2d98ca9652fb","Type":"ContainerStarted","Data":"d45f5e319c3dceb4e436113e547d38de96be917544dcbcff81ca404ff147b315"} Jul 18 11:53:48.819248 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:48.819171 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" event={"ID":"110995e9-ae10-4681-8274-2d98ca9652fb","Type":"ContainerStarted","Data":"96c5e310099b6b7fef927020ce9845dd22b294e9c850f263743fc28a7a7da0b3"} Jul 18 11:53:48.819587 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:48.819294 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:53:48.837594 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:48.837539 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" podStartSLOduration=1.427005684 podStartE2EDuration="2.83752192s" totalImagesPullingTime="1.410516236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:53:48.836148032 +0000 UTC m=+297.908207546" watchObservedRunningTime="2026-07-18 11:53:48.83752192 +0000 UTC m=+297.909581423" Jul 18 11:53:51.308647 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:51.308605 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:53:51.311812 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:51.311787 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:53:51.315966 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:51.315944 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 11:53:52.933006 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.932959 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk"] Jul 18 11:53:52.936476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.936454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:52.938680 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.938654 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 11:53:52.938990 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.938974 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 11:53:52.939097 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.939068 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 11:53:52.939385 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.939367 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 11:53:52.939425 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.939393 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m7xcv\"" Jul 18 11:53:52.961251 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:52.961224 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk"] Jul 18 11:53:53.046386 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.046349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4xsv\" (UniqueName: \"kubernetes.io/projected/bc49d736-1e38-4146-bf63-53333de54985-kube-api-access-l4xsv\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.046386 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.046387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-tmp\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.046590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.046457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.046590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.046493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.046590 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.046545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-manifests\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.147830 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.147788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xsv\" (UniqueName: \"kubernetes.io/projected/bc49d736-1e38-4146-bf63-53333de54985-kube-api-access-l4xsv\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.147830 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.147831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-tmp\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.148091 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.147866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.148091 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.147881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.148091 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.147914 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-manifests\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.148327 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.148272 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-manifests\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.150183 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.150162 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc49d736-1e38-4146-bf63-53333de54985-tmp\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.150427 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.150409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.150488 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.150443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc49d736-1e38-4146-bf63-53333de54985-webhook-cert\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.156415 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.156387 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4xsv\" (UniqueName: \"kubernetes.io/projected/bc49d736-1e38-4146-bf63-53333de54985-kube-api-access-l4xsv\") pod \"opendatahub-operator-controller-manager-fbb59dc99-rwbbk\" (UID: \"bc49d736-1e38-4146-bf63-53333de54985\") " pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.246665 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.246576 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:53.381077 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.380970 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk"] Jul 18 11:53:53.383486 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:53:53.383458 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc49d736_1e38_4146_bf63_53333de54985.slice/crio-439abddbd7405b67b4545496c7dd8ab646b177007f5076d9a77ca5a01d78d272 WatchSource:0}: Error finding container 439abddbd7405b67b4545496c7dd8ab646b177007f5076d9a77ca5a01d78d272: Status 404 returned error can't find the container with id 439abddbd7405b67b4545496c7dd8ab646b177007f5076d9a77ca5a01d78d272 Jul 18 11:53:53.385367 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.385346 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:53:53.848528 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:53.848484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" event={"ID":"bc49d736-1e38-4146-bf63-53333de54985","Type":"ContainerStarted","Data":"439abddbd7405b67b4545496c7dd8ab646b177007f5076d9a77ca5a01d78d272"} Jul 18 11:53:56.865665 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:56.865632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" event={"ID":"bc49d736-1e38-4146-bf63-53333de54985","Type":"ContainerStarted","Data":"e913358f79608433ec6542fdc39cfd51f123c4edf69e09a677d8f133bc8de0a1"} Jul 18 11:53:57.871615 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:57.871578 2578 generic.go:356] "Generic (PLEG): container finished" podID="bc49d736-1e38-4146-bf63-53333de54985" containerID="e913358f79608433ec6542fdc39cfd51f123c4edf69e09a677d8f133bc8de0a1" exitCode=0 Jul 18 11:53:57.872042 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:57.871660 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" event={"ID":"bc49d736-1e38-4146-bf63-53333de54985","Type":"ContainerDied","Data":"e913358f79608433ec6542fdc39cfd51f123c4edf69e09a677d8f133bc8de0a1"} Jul 18 11:53:58.877841 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:58.877803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" event={"ID":"bc49d736-1e38-4146-bf63-53333de54985","Type":"ContainerStarted","Data":"836067fd39248e351a7801858d818a10ddd70b0ab337dff2833c2b92a5b48f30"} Jul 18 11:53:58.878255 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:58.877952 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:53:58.904906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:58.904768 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" podStartSLOduration=2.677428112 podStartE2EDuration="6.904735047s" totalImagesPullingTime="3.196810172s" totalInitContainerRuntime="1.030496763s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:53:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:53:58.9013946 +0000 UTC m=+307.973454101" watchObservedRunningTime="2026-07-18 11:53:58.904735047 +0000 UTC m=+307.976794549" Jul 18 11:53:59.825543 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:53:59.825513 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77c8bd676b-xg55x" Jul 18 11:54:09.885079 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:09.885033 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fbb59dc99-rwbbk" Jul 18 11:54:22.990447 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.990416 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk"] Jul 18 11:54:22.993875 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.993855 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:22.996912 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.996876 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 11:54:22.996912 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.996882 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2n9js\"" Jul 18 11:54:22.997607 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.997589 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 11:54:22.997607 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.997599 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 18 11:54:22.997724 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:22.997597 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 18 11:54:23.017926 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.017892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk"] Jul 18 11:54:23.097765 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.097736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj59w\" (UniqueName: \"kubernetes.io/projected/dfa2a5d1-8057-4640-ae65-648fd313b181-kube-api-access-bj59w\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.097909 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.097777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.097909 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.097796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfa2a5d1-8057-4640-ae65-648fd313b181-tmp\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.198831 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.198804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj59w\" (UniqueName: \"kubernetes.io/projected/dfa2a5d1-8057-4640-ae65-648fd313b181-kube-api-access-bj59w\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.198968 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.198852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.198968 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.198870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfa2a5d1-8057-4640-ae65-648fd313b181-tmp\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.199066 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:54:23.198961 2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 18 11:54:23.199066 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:54:23.199033 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs podName:dfa2a5d1-8057-4640-ae65-648fd313b181 nodeName:}" failed. No retries permitted until 2026-07-18 11:54:23.69901419 +0000 UTC m=+332.771073671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs") pod "kube-auth-proxy-677848b8b7-b9vvk" (UID: "dfa2a5d1-8057-4640-ae65-648fd313b181") : secret "kube-auth-proxy-tls" not found Jul 18 11:54:23.201019 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.201002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfa2a5d1-8057-4640-ae65-648fd313b181-tmp\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.210488 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.210468 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj59w\" (UniqueName: \"kubernetes.io/projected/dfa2a5d1-8057-4640-ae65-648fd313b181-kube-api-access-bj59w\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.704575 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.704539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.706855 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.706835 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2a5d1-8057-4640-ae65-648fd313b181-tls-certs\") pod \"kube-auth-proxy-677848b8b7-b9vvk\" (UID: \"dfa2a5d1-8057-4640-ae65-648fd313b181\") " pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:23.903888 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:23.903854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" Jul 18 11:54:24.065586 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:24.065561 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk"] Jul 18 11:54:24.067720 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:54:24.067687 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa2a5d1_8057_4640_ae65_648fd313b181.slice/crio-1aa8ef58660022eb63b3c6bdca78ed96746d5922a60d7a76b30b68ce53a3fd74 WatchSource:0}: Error finding container 1aa8ef58660022eb63b3c6bdca78ed96746d5922a60d7a76b30b68ce53a3fd74: Status 404 returned error can't find the container with id 1aa8ef58660022eb63b3c6bdca78ed96746d5922a60d7a76b30b68ce53a3fd74 Jul 18 11:54:25.000415 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:25.000381 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" event={"ID":"dfa2a5d1-8057-4640-ae65-648fd313b181","Type":"ContainerStarted","Data":"1aa8ef58660022eb63b3c6bdca78ed96746d5922a60d7a76b30b68ce53a3fd74"} Jul 18 11:54:27.015488 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:27.015401 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" event={"ID":"dfa2a5d1-8057-4640-ae65-648fd313b181","Type":"ContainerStarted","Data":"6b5d1e07d3a92c02dafd992520f0a51422a1d98d01763ab118607431533e81c1"} Jul 18 11:54:27.032153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:54:27.032102 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-677848b8b7-b9vvk" podStartSLOduration=2.431075699 podStartE2EDuration="5.032086699s" totalImagesPullingTime="2.601011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:54:27.029595552 +0000 UTC m=+336.101655057" watchObservedRunningTime="2026-07-18 11:54:27.032086699 +0000 UTC m=+336.104146201" Jul 18 11:55:43.041588 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.041548 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw"] Jul 18 11:55:43.045229 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.045207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.047474 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.047456 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 11:55:43.047578 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.047545 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 11:55:43.048153 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.048139 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mrrx5\"" Jul 18 11:55:43.051674 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.051652 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw"] Jul 18 11:55:43.062340 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.062314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6779\" (UniqueName: \"kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.062456 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.062353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.062456 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.062404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.141565 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.141534 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6"] Jul 18 11:55:43.144985 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.144967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.152438 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.152417 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6"] Jul 18 11:55:43.162882 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.162855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn4nf\" (UniqueName: \"kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.163013 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.162891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.163013 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.162928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.163013 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.162977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6779\" (UniqueName: \"kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.163202 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.163067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.163202 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.163114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.163302 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.163273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.163388 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.163372 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.170782 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.170752 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6779\" (UniqueName: \"kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.263467 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.263435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.263649 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.263492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn4nf\" (UniqueName: \"kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.263649 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.263517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.263902 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.263881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.263970 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.263892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.271588 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.271566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn4nf\" (UniqueName: \"kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.354871 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.354790 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:43.442188 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.442157 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7"] Jul 18 11:55:43.447573 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.447548 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.454434 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.454414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:43.454800 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.454688 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7"] Jul 18 11:55:43.465522 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.465495 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.465731 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.465687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.465805 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.465724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfvh\" (UniqueName: \"kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.495286 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.495265 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw"] Jul 18 11:55:43.497801 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:55:43.497776 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9ae344_5a95_48f9_b0a9_a9a650ce9bad.slice/crio-cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1 WatchSource:0}: Error finding container cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1: Status 404 returned error can't find the container with id cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1 Jul 18 11:55:43.567169 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.567136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.567328 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.567189 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfvh\" (UniqueName: \"kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.567328 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.567267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.567580 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.567561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.567663 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.567643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.575811 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.575781 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfvh\" (UniqueName: \"kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.592364 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.592341 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6"] Jul 18 11:55:43.596184 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:55:43.596156 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d822b8d_0156_4f21_b5c7_714f83053334.slice/crio-86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13 WatchSource:0}: Error finding container 86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13: Status 404 returned error can't find the container with id 86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13 Jul 18 11:55:43.759771 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.759746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:43.841303 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.841263 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h"] Jul 18 11:55:43.844929 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.844878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.851255 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.851231 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h"] Jul 18 11:55:43.869726 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.869669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.869726 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.869701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khcpv\" (UniqueName: \"kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.869860 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.869744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.894169 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.894142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7"] Jul 18 11:55:43.896474 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:55:43.896448 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod199bca4c_fab0_4473_8a56_77bf78e51f2a.slice/crio-72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e WatchSource:0}: Error finding container 72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e: Status 404 returned error can't find the container with id 72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e Jul 18 11:55:43.971164 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.971139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.971300 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.971198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.971300 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.971229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khcpv\" (UniqueName: \"kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.971565 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.971544 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.971624 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.971573 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:43.978804 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:43.978773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khcpv\" (UniqueName: \"kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:44.160262 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.160171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:44.287629 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.287603 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h"] Jul 18 11:55:44.290371 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:55:44.290345 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f27f110_5bc4_4f3e_bf0b_8b2662d15e82.slice/crio-c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79 WatchSource:0}: Error finding container c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79: Status 404 returned error can't find the container with id c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79 Jul 18 11:55:44.360218 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.360189 2578 generic.go:356] "Generic (PLEG): container finished" podID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerID="223eef6dfe05a346dfc05a419c6eae460219e37019e56b21afe7a6f78fd83ba6" exitCode=0 Jul 18 11:55:44.360336 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.360242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" event={"ID":"199bca4c-fab0-4473-8a56-77bf78e51f2a","Type":"ContainerDied","Data":"223eef6dfe05a346dfc05a419c6eae460219e37019e56b21afe7a6f78fd83ba6"} Jul 18 11:55:44.360336 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.360299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" event={"ID":"199bca4c-fab0-4473-8a56-77bf78e51f2a","Type":"ContainerStarted","Data":"72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e"} Jul 18 11:55:44.361922 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.361901 2578 generic.go:356] "Generic (PLEG): container finished" podID="3d822b8d-0156-4f21-b5c7-714f83053334" containerID="6e2c951ac9f24f72f2e4f5f8937f55e6bf9eab34c5da33c28c6589e539f7d7f5" exitCode=0 Jul 18 11:55:44.362016 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.361937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" event={"ID":"3d822b8d-0156-4f21-b5c7-714f83053334","Type":"ContainerDied","Data":"6e2c951ac9f24f72f2e4f5f8937f55e6bf9eab34c5da33c28c6589e539f7d7f5"} Jul 18 11:55:44.362016 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.361998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" event={"ID":"3d822b8d-0156-4f21-b5c7-714f83053334","Type":"ContainerStarted","Data":"86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13"} Jul 18 11:55:44.363657 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.363620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerStarted","Data":"50a9506a154d671f23304431c9f508a473365140adb293592eac6e7394281c80"} Jul 18 11:55:44.363764 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.363664 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerStarted","Data":"c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79"} Jul 18 11:55:44.365264 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.365242 2578 generic.go:356] "Generic (PLEG): container finished" podID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerID="f65c7987d46f165dc0c8c36e9deb3a9acac112020b520dc4fb005ad407201c34" exitCode=0 Jul 18 11:55:44.365368 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.365277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" event={"ID":"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad","Type":"ContainerDied","Data":"f65c7987d46f165dc0c8c36e9deb3a9acac112020b520dc4fb005ad407201c34"} Jul 18 11:55:44.365368 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:44.365317 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" event={"ID":"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad","Type":"ContainerStarted","Data":"cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1"} Jul 18 11:55:45.372109 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.372075 2578 generic.go:356] "Generic (PLEG): container finished" podID="3d822b8d-0156-4f21-b5c7-714f83053334" containerID="b08ff77b5e247e734764c021c325deaf3b1a39a63b21068f0ac9587a4ca1f0a5" exitCode=0 Jul 18 11:55:45.372464 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.372159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" event={"ID":"3d822b8d-0156-4f21-b5c7-714f83053334","Type":"ContainerDied","Data":"b08ff77b5e247e734764c021c325deaf3b1a39a63b21068f0ac9587a4ca1f0a5"} Jul 18 11:55:45.373813 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.373791 2578 generic.go:356] "Generic (PLEG): container finished" podID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerID="50a9506a154d671f23304431c9f508a473365140adb293592eac6e7394281c80" exitCode=0 Jul 18 11:55:45.373904 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.373886 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerDied","Data":"50a9506a154d671f23304431c9f508a473365140adb293592eac6e7394281c80"} Jul 18 11:55:45.375819 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.375792 2578 generic.go:356] "Generic (PLEG): container finished" podID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerID="806a6cbf1644235c319cc1769eb77bba6d8a05299cb109b06953a79f66e3ad2c" exitCode=0 Jul 18 11:55:45.375906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:45.375845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" event={"ID":"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad","Type":"ContainerDied","Data":"806a6cbf1644235c319cc1769eb77bba6d8a05299cb109b06953a79f66e3ad2c"} Jul 18 11:55:46.382578 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.382486 2578 generic.go:356] "Generic (PLEG): container finished" podID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerID="62a0ce11b47bf7aeceeb622ce833af06bba5ca034b13a684322746fd25cd1810" exitCode=0 Jul 18 11:55:46.382578 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.382532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" event={"ID":"199bca4c-fab0-4473-8a56-77bf78e51f2a","Type":"ContainerDied","Data":"62a0ce11b47bf7aeceeb622ce833af06bba5ca034b13a684322746fd25cd1810"} Jul 18 11:55:46.384777 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.384755 2578 generic.go:356] "Generic (PLEG): container finished" podID="3d822b8d-0156-4f21-b5c7-714f83053334" containerID="3749f8ea19c68806957edfc4203fc8efd94d5c52997b0265351011e98d0e3231" exitCode=0 Jul 18 11:55:46.384878 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.384838 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" event={"ID":"3d822b8d-0156-4f21-b5c7-714f83053334","Type":"ContainerDied","Data":"3749f8ea19c68806957edfc4203fc8efd94d5c52997b0265351011e98d0e3231"} Jul 18 11:55:46.386606 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.386585 2578 generic.go:356] "Generic (PLEG): container finished" podID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerID="77a53083ee81bddaee044776eb20a668cb3290ed6b29f0b33ff748436ec6d8fa" exitCode=0 Jul 18 11:55:46.386737 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.386665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerDied","Data":"77a53083ee81bddaee044776eb20a668cb3290ed6b29f0b33ff748436ec6d8fa"} Jul 18 11:55:46.388953 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.388933 2578 generic.go:356] "Generic (PLEG): container finished" podID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerID="21e53a3620d55020cce76870dc8e24998bc56818375407b17df41a53fbcb6fc5" exitCode=0 Jul 18 11:55:46.389066 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:46.388990 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" event={"ID":"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad","Type":"ContainerDied","Data":"21e53a3620d55020cce76870dc8e24998bc56818375407b17df41a53fbcb6fc5"} Jul 18 11:55:47.402311 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.402269 2578 generic.go:356] "Generic (PLEG): container finished" podID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerID="0c88fb506360484e99c21dad746b16146f8291e445864ff190defaebc9399ba3" exitCode=0 Jul 18 11:55:47.402759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.402349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerDied","Data":"0c88fb506360484e99c21dad746b16146f8291e445864ff190defaebc9399ba3"} Jul 18 11:55:47.404982 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.404954 2578 generic.go:356] "Generic (PLEG): container finished" podID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerID="2ed6761e8b19b8a4b51f152031b6e3979cba6cdbfe3fef59a4ace75de1d6f2d2" exitCode=0 Jul 18 11:55:47.405094 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.405022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" event={"ID":"199bca4c-fab0-4473-8a56-77bf78e51f2a","Type":"ContainerDied","Data":"2ed6761e8b19b8a4b51f152031b6e3979cba6cdbfe3fef59a4ace75de1d6f2d2"} Jul 18 11:55:47.554899 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.554865 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:47.561808 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.561785 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:47.603794 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.603766 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle\") pod \"3d822b8d-0156-4f21-b5c7-714f83053334\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " Jul 18 11:55:47.603946 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.603814 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util\") pod \"3d822b8d-0156-4f21-b5c7-714f83053334\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " Jul 18 11:55:47.603946 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.603856 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6779\" (UniqueName: \"kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779\") pod \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " Jul 18 11:55:47.603946 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.603905 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util\") pod \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " Jul 18 11:55:47.604173 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.604032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn4nf\" (UniqueName: \"kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf\") pod \"3d822b8d-0156-4f21-b5c7-714f83053334\" (UID: \"3d822b8d-0156-4f21-b5c7-714f83053334\") " Jul 18 11:55:47.604173 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.604111 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle\") pod \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\" (UID: \"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad\") " Jul 18 11:55:47.604519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.604376 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle" (OuterVolumeSpecName: "bundle") pod "3d822b8d-0156-4f21-b5c7-714f83053334" (UID: "3d822b8d-0156-4f21-b5c7-714f83053334"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:47.604878 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.604833 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle" (OuterVolumeSpecName: "bundle") pod "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" (UID: "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:47.606237 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.606215 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779" (OuterVolumeSpecName: "kube-api-access-r6779") pod "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" (UID: "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad"). InnerVolumeSpecName "kube-api-access-r6779". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:55:47.606502 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.606485 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf" (OuterVolumeSpecName: "kube-api-access-dn4nf") pod "3d822b8d-0156-4f21-b5c7-714f83053334" (UID: "3d822b8d-0156-4f21-b5c7-714f83053334"). InnerVolumeSpecName "kube-api-access-dn4nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:55:47.609596 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.609572 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util" (OuterVolumeSpecName: "util") pod "3d822b8d-0156-4f21-b5c7-714f83053334" (UID: "3d822b8d-0156-4f21-b5c7-714f83053334"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:47.610985 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.610960 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util" (OuterVolumeSpecName: "util") pod "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" (UID: "9b9ae344-5a95-48f9-b0a9-a9a650ce9bad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:47.705454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705423 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn4nf\" (UniqueName: \"kubernetes.io/projected/3d822b8d-0156-4f21-b5c7-714f83053334-kube-api-access-dn4nf\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:47.705454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705449 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:47.705454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705457 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:47.705454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705465 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d822b8d-0156-4f21-b5c7-714f83053334-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:47.705689 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705473 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6779\" (UniqueName: \"kubernetes.io/projected/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-kube-api-access-r6779\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:47.705689 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:47.705482 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b9ae344-5a95-48f9-b0a9-a9a650ce9bad-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.412083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.412039 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" Jul 18 11:55:48.412083 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.412066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw" event={"ID":"9b9ae344-5a95-48f9-b0a9-a9a650ce9bad","Type":"ContainerDied","Data":"cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1"} Jul 18 11:55:48.412568 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.412109 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbeb8456005539d73760dcf0f1a2d0aa81c0b47c0ad85e8c9f89b42e11298bc1" Jul 18 11:55:48.414085 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.414067 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" Jul 18 11:55:48.414213 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.414040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6" event={"ID":"3d822b8d-0156-4f21-b5c7-714f83053334","Type":"ContainerDied","Data":"86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13"} Jul 18 11:55:48.414213 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.414122 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86f6eefc4e2187bdeb4c7d7b57a28984015642ae96515057e973633752257f13" Jul 18 11:55:48.550424 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.550394 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:48.558828 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.558810 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:48.612700 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612673 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khcpv\" (UniqueName: \"kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv\") pod \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " Jul 18 11:55:48.612863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612727 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clfvh\" (UniqueName: \"kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh\") pod \"199bca4c-fab0-4473-8a56-77bf78e51f2a\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " Jul 18 11:55:48.612863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612761 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util\") pod \"199bca4c-fab0-4473-8a56-77bf78e51f2a\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " Jul 18 11:55:48.612863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612789 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util\") pod \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " Jul 18 11:55:48.612863 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612840 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle\") pod \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\" (UID: \"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82\") " Jul 18 11:55:48.613089 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.612871 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle\") pod \"199bca4c-fab0-4473-8a56-77bf78e51f2a\" (UID: \"199bca4c-fab0-4473-8a56-77bf78e51f2a\") " Jul 18 11:55:48.613420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.613388 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle" (OuterVolumeSpecName: "bundle") pod "199bca4c-fab0-4473-8a56-77bf78e51f2a" (UID: "199bca4c-fab0-4473-8a56-77bf78e51f2a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:48.613519 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.613474 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle" (OuterVolumeSpecName: "bundle") pod "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" (UID: "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:48.614957 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.614928 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh" (OuterVolumeSpecName: "kube-api-access-clfvh") pod "199bca4c-fab0-4473-8a56-77bf78e51f2a" (UID: "199bca4c-fab0-4473-8a56-77bf78e51f2a"). InnerVolumeSpecName "kube-api-access-clfvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:55:48.615340 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.615320 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv" (OuterVolumeSpecName: "kube-api-access-khcpv") pod "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" (UID: "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82"). InnerVolumeSpecName "kube-api-access-khcpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:55:48.618749 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.618706 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util" (OuterVolumeSpecName: "util") pod "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" (UID: "2f27f110-5bc4-4f3e-bf0b-8b2662d15e82"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:48.621040 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.621017 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util" (OuterVolumeSpecName: "util") pod "199bca4c-fab0-4473-8a56-77bf78e51f2a" (UID: "199bca4c-fab0-4473-8a56-77bf78e51f2a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 11:55:48.713764 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713719 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khcpv\" (UniqueName: \"kubernetes.io/projected/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-kube-api-access-khcpv\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.713764 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713768 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfvh\" (UniqueName: \"kubernetes.io/projected/199bca4c-fab0-4473-8a56-77bf78e51f2a-kube-api-access-clfvh\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.713973 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713785 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.713973 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713801 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-util\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.713973 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713814 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f27f110-5bc4-4f3e-bf0b-8b2662d15e82-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:48.713973 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:48.713826 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/199bca4c-fab0-4473-8a56-77bf78e51f2a-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:55:49.422537 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.422511 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" Jul 18 11:55:49.422537 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.422523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h" event={"ID":"2f27f110-5bc4-4f3e-bf0b-8b2662d15e82","Type":"ContainerDied","Data":"c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79"} Jul 18 11:55:49.422928 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.422567 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c08be6655e9eb5534828dc1602825c3eee2a111de7b2c84776735b0406be3f79" Jul 18 11:55:49.424609 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.424588 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" Jul 18 11:55:49.424706 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.424594 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7" event={"ID":"199bca4c-fab0-4473-8a56-77bf78e51f2a","Type":"ContainerDied","Data":"72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e"} Jul 18 11:55:49.424767 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:55:49.424703 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72e553a1549885c9e7df8f776c9dc20e75fc3822e43ddd9b1f31eb6d5b41ae0e" Jul 18 11:56:18.949780 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.949699 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4"] Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950005 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950016 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950027 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950033 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950067 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950073 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950082 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950087 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950098 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950103 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950112 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950116 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950122 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950128 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950134 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950139 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950145 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950150 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950155 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950160 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950166 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950172 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="util" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950177 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950181 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="pull" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950228 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f27f110-5bc4-4f3e-bf0b-8b2662d15e82" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950236 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9ae344-5a95-48f9-b0a9-a9a650ce9bad" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950241 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="199bca4c-fab0-4473-8a56-77bf78e51f2a" containerName="extract" Jul 18 11:56:18.950246 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.950247 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d822b8d-0156-4f21-b5c7-714f83053334" containerName="extract" Jul 18 11:56:18.953279 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.953263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:18.955459 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.955439 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 18 11:56:18.955716 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.955696 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 11:56:18.955765 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.955696 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 11:56:18.955765 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.955738 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 18 11:56:18.956344 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.956330 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mrrx5\"" Jul 18 11:56:18.965691 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:18.965660 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4"] Jul 18 11:56:19.070605 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.070572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkqgk\" (UniqueName: \"kubernetes.io/projected/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-kube-api-access-jkqgk\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.070774 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.070611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.070774 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.070680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.171436 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.171401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.171595 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.171453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkqgk\" (UniqueName: \"kubernetes.io/projected/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-kube-api-access-jkqgk\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.171595 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.171485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.172210 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.172185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.173850 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.173830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.187310 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.187282 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkqgk\" (UniqueName: \"kubernetes.io/projected/8f001dbb-89c2-49c0-bfc2-4ba18a55cf24-kube-api-access-jkqgk\") pod \"kuadrant-console-plugin-66b5774985-tw8l4\" (UID: \"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.268554 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.268473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" Jul 18 11:56:19.397492 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.397454 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4"] Jul 18 11:56:19.400931 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:56:19.400906 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f001dbb_89c2_49c0_bfc2_4ba18a55cf24.slice/crio-9f5aab693a631c9881d83e181c161d30d2cf4f5cc2d53fab1dfc82f7af64d93a WatchSource:0}: Error finding container 9f5aab693a631c9881d83e181c161d30d2cf4f5cc2d53fab1dfc82f7af64d93a: Status 404 returned error can't find the container with id 9f5aab693a631c9881d83e181c161d30d2cf4f5cc2d53fab1dfc82f7af64d93a Jul 18 11:56:19.573093 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:19.573002 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" event={"ID":"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24","Type":"ContainerStarted","Data":"9f5aab693a631c9881d83e181c161d30d2cf4f5cc2d53fab1dfc82f7af64d93a"} Jul 18 11:56:40.697028 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:40.696992 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" event={"ID":"8f001dbb-89c2-49c0-bfc2-4ba18a55cf24","Type":"ContainerStarted","Data":"154bdb634efc73d441b6bd90f1467a710e0aee813e1d5d16ab7b6f61998a07ae"} Jul 18 11:56:40.713271 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:40.713223 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tw8l4" podStartSLOduration=2.092736371 podStartE2EDuration="22.713208469s" totalImagesPullingTime="20.620472098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:56:40.712067212 +0000 UTC m=+469.784126710" watchObservedRunningTime="2026-07-18 11:56:40.713208469 +0000 UTC m=+469.785268015" Jul 18 11:56:42.006156 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.006120 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 11:56:42.010511 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.010493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.012894 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.012873 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qvmch\"" Jul 18 11:56:42.031773 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.031657 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 11:56:42.059696 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.059668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.059831 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.059705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxb7l\" (UniqueName: \"kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.160680 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.160644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.160823 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.160690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxb7l\" (UniqueName: \"kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.161036 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.161018 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.168963 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.168940 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxb7l\" (UniqueName: \"kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l\") pod \"kuadrant-operator-controller-manager-86878c769f-l85g9\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.320024 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.319934 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:42.455467 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.455440 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 11:56:42.457747 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:56:42.457719 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c462e89_cc83_485e_a0b6_5b46f9855b59.slice/crio-f9c399c366df934ab712d02ba19fd8b68599488c9898eae3703afdb1a5b41442 WatchSource:0}: Error finding container f9c399c366df934ab712d02ba19fd8b68599488c9898eae3703afdb1a5b41442: Status 404 returned error can't find the container with id f9c399c366df934ab712d02ba19fd8b68599488c9898eae3703afdb1a5b41442 Jul 18 11:56:42.707793 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:42.707761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" event={"ID":"4c462e89-cc83-485e-a0b6-5b46f9855b59","Type":"ContainerStarted","Data":"f9c399c366df934ab712d02ba19fd8b68599488c9898eae3703afdb1a5b41442"} Jul 18 11:56:47.736869 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:47.736832 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" event={"ID":"4c462e89-cc83-485e-a0b6-5b46f9855b59","Type":"ContainerStarted","Data":"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2"} Jul 18 11:56:47.737276 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:47.736967 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:56:47.759114 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:47.759064 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" podStartSLOduration=2.458514015 podStartE2EDuration="6.759031674s" totalImagesPullingTime="4.300517659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:56:47.757751573 +0000 UTC m=+476.829811076" watchObservedRunningTime="2026-07-18 11:56:47.759031674 +0000 UTC m=+476.831091177" Jul 18 11:56:58.744281 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:56:58.744247 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 11:57:07.016670 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.016641 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bf56d4ccb-s7jtc"] Jul 18 11:57:07.103281 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.103245 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf56d4ccb-s7jtc"] Jul 18 11:57:07.103437 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.103360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.170997 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.170962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-trusted-ca-bundle\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-console-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-oauth-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqns2\" (UniqueName: \"kubernetes.io/projected/24686a53-3069-4793-82b7-ca71a56cecf4-kube-api-access-gqns2\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-oauth-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-service-ca\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.171190 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.171181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.271889 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.271796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272096 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.271956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-trusted-ca-bundle\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272096 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.271991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-console-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272096 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-oauth-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272096 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqns2\" (UniqueName: \"kubernetes.io/projected/24686a53-3069-4793-82b7-ca71a56cecf4-kube-api-access-gqns2\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272324 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-oauth-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272324 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-service-ca\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272781 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272756 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-console-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272900 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272865 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-oauth-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272955 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-service-ca\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.272991 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.272949 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24686a53-3069-4793-82b7-ca71a56cecf4-trusted-ca-bundle\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.274962 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.274934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-oauth-config\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.275075 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.275022 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24686a53-3069-4793-82b7-ca71a56cecf4-console-serving-cert\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.279821 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.279799 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqns2\" (UniqueName: \"kubernetes.io/projected/24686a53-3069-4793-82b7-ca71a56cecf4-kube-api-access-gqns2\") pod \"console-bf56d4ccb-s7jtc\" (UID: \"24686a53-3069-4793-82b7-ca71a56cecf4\") " pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.412640 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.412612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:07.562578 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.562548 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf56d4ccb-s7jtc"] Jul 18 11:57:07.564697 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:57:07.564667 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24686a53_3069_4793_82b7_ca71a56cecf4.slice/crio-9538f497294b665cd41293c9eb64d833877798a9b772b481518425da847b5184 WatchSource:0}: Error finding container 9538f497294b665cd41293c9eb64d833877798a9b772b481518425da847b5184: Status 404 returned error can't find the container with id 9538f497294b665cd41293c9eb64d833877798a9b772b481518425da847b5184 Jul 18 11:57:07.842017 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.841928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf56d4ccb-s7jtc" event={"ID":"24686a53-3069-4793-82b7-ca71a56cecf4","Type":"ContainerStarted","Data":"b7e63391ff06e8d4a8996ec3e6e4278dd0c3a87f7932e9008b559801bea3aa69"} Jul 18 11:57:07.842017 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.841979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf56d4ccb-s7jtc" event={"ID":"24686a53-3069-4793-82b7-ca71a56cecf4","Type":"ContainerStarted","Data":"9538f497294b665cd41293c9eb64d833877798a9b772b481518425da847b5184"} Jul 18 11:57:07.859869 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:07.859812 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bf56d4ccb-s7jtc" podStartSLOduration=1.859797704 podStartE2EDuration="1.859797704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:57:07.858795524 +0000 UTC m=+496.930855028" watchObservedRunningTime="2026-07-18 11:57:07.859797704 +0000 UTC m=+496.931857207" Jul 18 11:57:17.412903 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:17.412864 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:17.413413 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:17.412925 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:17.417909 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:17.417885 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:17.895129 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:17.895101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bf56d4ccb-s7jtc" Jul 18 11:57:17.938801 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:17.938768 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:57:19.335607 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.335565 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:19.339836 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.339812 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:19.342013 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.341985 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xlzqh\"" Jul 18 11:57:19.345846 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.345805 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:19.475174 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.475137 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpz5n\" (UniqueName: \"kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n\") pod \"authorino-58cd955675-cbllk\" (UID: \"efaeb346-f475-4d53-aabc-bcd572042c91\") " pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:19.576495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.576461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpz5n\" (UniqueName: \"kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n\") pod \"authorino-58cd955675-cbllk\" (UID: \"efaeb346-f475-4d53-aabc-bcd572042c91\") " pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:19.584453 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.584422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpz5n\" (UniqueName: \"kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n\") pod \"authorino-58cd955675-cbllk\" (UID: \"efaeb346-f475-4d53-aabc-bcd572042c91\") " pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:19.650915 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.650846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:19.775505 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.775476 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:19.776818 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:57:19.776790 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefaeb346_f475_4d53_aabc_bcd572042c91.slice/crio-ef538a02024851b662688b5d9f3cc388940c39cda83b55fcc47b31f09057a9d7 WatchSource:0}: Error finding container ef538a02024851b662688b5d9f3cc388940c39cda83b55fcc47b31f09057a9d7: Status 404 returned error can't find the container with id ef538a02024851b662688b5d9f3cc388940c39cda83b55fcc47b31f09057a9d7 Jul 18 11:57:19.902810 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:19.902731 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cbllk" event={"ID":"efaeb346-f475-4d53-aabc-bcd572042c91","Type":"ContainerStarted","Data":"ef538a02024851b662688b5d9f3cc388940c39cda83b55fcc47b31f09057a9d7"} Jul 18 11:57:22.923745 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:22.923652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cbllk" event={"ID":"efaeb346-f475-4d53-aabc-bcd572042c91","Type":"ContainerStarted","Data":"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c"} Jul 18 11:57:22.938597 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:22.938547 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-cbllk" podStartSLOduration=1.045688467 podStartE2EDuration="3.93853175s" totalImagesPullingTime="2.892843283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:57:22.937031215 +0000 UTC m=+512.009090721" watchObservedRunningTime="2026-07-18 11:57:22.93853175 +0000 UTC m=+512.010591253" Jul 18 11:57:22.971495 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:22.971466 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:24.934455 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:24.934419 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-cbllk" podUID="efaeb346-f475-4d53-aabc-bcd572042c91" containerName="authorino" containerID="cri-o://563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c" gracePeriod=30 Jul 18 11:57:25.178767 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.178744 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:25.327327 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.327240 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpz5n\" (UniqueName: \"kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n\") pod \"efaeb346-f475-4d53-aabc-bcd572042c91\" (UID: \"efaeb346-f475-4d53-aabc-bcd572042c91\") " Jul 18 11:57:25.329268 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.329236 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n" (OuterVolumeSpecName: "kube-api-access-dpz5n") pod "efaeb346-f475-4d53-aabc-bcd572042c91" (UID: "efaeb346-f475-4d53-aabc-bcd572042c91"). InnerVolumeSpecName "kube-api-access-dpz5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:57:25.427941 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.427917 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpz5n\" (UniqueName: \"kubernetes.io/projected/efaeb346-f475-4d53-aabc-bcd572042c91-kube-api-access-dpz5n\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:25.946476 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.946440 2578 generic.go:356] "Generic (PLEG): container finished" podID="efaeb346-f475-4d53-aabc-bcd572042c91" containerID="563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c" exitCode=0 Jul 18 11:57:25.946883 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.946488 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cbllk" Jul 18 11:57:25.946883 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.946526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cbllk" event={"ID":"efaeb346-f475-4d53-aabc-bcd572042c91","Type":"ContainerDied","Data":"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c"} Jul 18 11:57:25.946883 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.946565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cbllk" event={"ID":"efaeb346-f475-4d53-aabc-bcd572042c91","Type":"ContainerDied","Data":"ef538a02024851b662688b5d9f3cc388940c39cda83b55fcc47b31f09057a9d7"} Jul 18 11:57:25.946883 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.946581 2578 scope.go:117] "RemoveContainer" containerID="563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c" Jul 18 11:57:25.955075 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.955056 2578 scope.go:117] "RemoveContainer" containerID="563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c" Jul 18 11:57:25.955332 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:57:25.955313 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c\": container with ID starting with 563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c not found: ID does not exist" containerID="563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c" Jul 18 11:57:25.955379 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.955339 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c"} err="failed to get container status \"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c\": rpc error: code = NotFound desc = could not find container \"563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c\": container with ID starting with 563af5ef181821e3c48e0d6fbcfba5435a1801e11532cc933e6c899da4547c4c not found: ID does not exist" Jul 18 11:57:25.962849 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.962826 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:25.966845 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:25.966827 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cbllk"] Jul 18 11:57:27.412759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:27.412726 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efaeb346-f475-4d53-aabc-bcd572042c91" path="/var/lib/kubelet/pods/efaeb346-f475-4d53-aabc-bcd572042c91/volumes" Jul 18 11:57:42.960217 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:42.960178 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c9bcd7d47-nn565" podUID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" containerName="console" containerID="cri-o://4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b" gracePeriod=15 Jul 18 11:57:43.221980 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.221923 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:57:43.279454 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279419 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279474 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279498 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279523 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279559 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279582 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279628 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279626 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhk9c\" (UniqueName: \"kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c\") pod \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\" (UID: \"48cb69cc-3983-428e-96a7-0cefda5e3dbf\") " Jul 18 11:57:43.279978 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.279891 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca" (OuterVolumeSpecName: "service-ca") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:57:43.280069 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.280024 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:57:43.280136 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.280042 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config" (OuterVolumeSpecName: "console-config") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:57:43.280276 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.280252 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 11:57:43.281911 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.281883 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:57:43.282000 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.281902 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 11:57:43.282000 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.281932 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c" (OuterVolumeSpecName: "kube-api-access-bhk9c") pod "48cb69cc-3983-428e-96a7-0cefda5e3dbf" (UID: "48cb69cc-3983-428e-96a7-0cefda5e3dbf"). InnerVolumeSpecName "kube-api-access-bhk9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 11:57:43.380420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380391 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380414 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-oauth-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380420 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380425 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380631 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380434 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-serving-cert\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380631 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380443 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48cb69cc-3983-428e-96a7-0cefda5e3dbf-console-oauth-config\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380631 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380451 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48cb69cc-3983-428e-96a7-0cefda5e3dbf-trusted-ca-bundle\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:43.380631 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:43.380461 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhk9c\" (UniqueName: \"kubernetes.io/projected/48cb69cc-3983-428e-96a7-0cefda5e3dbf-kube-api-access-bhk9c\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 11:57:44.036678 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.036602 2578 generic.go:356] "Generic (PLEG): container finished" podID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" containerID="4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b" exitCode=0 Jul 18 11:57:44.036678 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.036646 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9bcd7d47-nn565" Jul 18 11:57:44.037134 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.036687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bcd7d47-nn565" event={"ID":"48cb69cc-3983-428e-96a7-0cefda5e3dbf","Type":"ContainerDied","Data":"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b"} Jul 18 11:57:44.037134 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.036740 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9bcd7d47-nn565" event={"ID":"48cb69cc-3983-428e-96a7-0cefda5e3dbf","Type":"ContainerDied","Data":"a965b1779f62e765a54276776a236a35ac4a46bd474e95d7617146c658d61770"} Jul 18 11:57:44.037134 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.036766 2578 scope.go:117] "RemoveContainer" containerID="4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b" Jul 18 11:57:44.045214 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.045195 2578 scope.go:117] "RemoveContainer" containerID="4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b" Jul 18 11:57:44.045468 ip-10-0-130-242 kubenswrapper[2578]: E0718 11:57:44.045445 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b\": container with ID starting with 4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b not found: ID does not exist" containerID="4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b" Jul 18 11:57:44.045547 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.045479 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b"} err="failed to get container status \"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b\": rpc error: code = NotFound desc = could not find container \"4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b\": container with ID starting with 4134aac1f7f4cd23cc15952584fc66d6f6d0f3682a0cb296d356d5f104743f6b not found: ID does not exist" Jul 18 11:57:44.078064 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.078011 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:57:44.088109 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:44.088081 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c9bcd7d47-nn565"] Jul 18 11:57:45.412040 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:57:45.412006 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" path="/var/lib/kubelet/pods/48cb69cc-3983-428e-96a7-0cefda5e3dbf/volumes" Jul 18 11:58:32.097024 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.096990 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4kgb4"] Jul 18 11:58:32.097482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097390 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" containerName="console" Jul 18 11:58:32.097482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097406 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" containerName="console" Jul 18 11:58:32.097482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097420 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efaeb346-f475-4d53-aabc-bcd572042c91" containerName="authorino" Jul 18 11:58:32.097482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097426 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="efaeb346-f475-4d53-aabc-bcd572042c91" containerName="authorino" Jul 18 11:58:32.097613 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097501 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="48cb69cc-3983-428e-96a7-0cefda5e3dbf" containerName="console" Jul 18 11:58:32.097613 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.097511 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="efaeb346-f475-4d53-aabc-bcd572042c91" containerName="authorino" Jul 18 11:58:32.100505 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.100484 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.105525 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.105503 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9qbwk\"" Jul 18 11:58:32.105636 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.105531 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 18 11:58:32.112906 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.112881 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4kgb4"] Jul 18 11:58:32.177411 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.177375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jqq4\" (UniqueName: \"kubernetes.io/projected/9365b5e2-a70e-42c0-8a7b-76b694c05504-kube-api-access-7jqq4\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.177569 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.177424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9365b5e2-a70e-42c0-8a7b-76b694c05504-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.278318 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.278288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jqq4\" (UniqueName: \"kubernetes.io/projected/9365b5e2-a70e-42c0-8a7b-76b694c05504-kube-api-access-7jqq4\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.278492 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.278343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9365b5e2-a70e-42c0-8a7b-76b694c05504-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.279091 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.279072 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9365b5e2-a70e-42c0-8a7b-76b694c05504-plugins-config-volume\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.287528 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.287508 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jqq4\" (UniqueName: \"kubernetes.io/projected/9365b5e2-a70e-42c0-8a7b-76b694c05504-kube-api-access-7jqq4\") pod \"payload-processing-5bf47cd988-4kgb4\" (UID: \"9365b5e2-a70e-42c0-8a7b-76b694c05504\") " pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.410650 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.410561 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:32.539735 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:32.539710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-4kgb4"] Jul 18 11:58:32.541377 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:58:32.541347 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9365b5e2_a70e_42c0_8a7b_76b694c05504.slice/crio-89ca3d1f74eb9a2106f5ca1e513112d786307bf0a5209c08c3ef8aa32362c7cf WatchSource:0}: Error finding container 89ca3d1f74eb9a2106f5ca1e513112d786307bf0a5209c08c3ef8aa32362c7cf: Status 404 returned error can't find the container with id 89ca3d1f74eb9a2106f5ca1e513112d786307bf0a5209c08c3ef8aa32362c7cf Jul 18 11:58:33.284875 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:33.284808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" event={"ID":"9365b5e2-a70e-42c0-8a7b-76b694c05504","Type":"ContainerStarted","Data":"89ca3d1f74eb9a2106f5ca1e513112d786307bf0a5209c08c3ef8aa32362c7cf"} Jul 18 11:58:34.291583 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:34.291489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" event={"ID":"9365b5e2-a70e-42c0-8a7b-76b694c05504","Type":"ContainerStarted","Data":"9aebf52e3b7185164aec8c12ac70488be6011e842ad4804fcecbd20174a82889"} Jul 18 11:58:34.291583 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:34.291561 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:34.307093 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:34.307033 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" podStartSLOduration=0.902127511 podStartE2EDuration="2.307017398s" totalImagesPullingTime="1.404889887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:58:34.305641773 +0000 UTC m=+583.377701277" watchObservedRunningTime="2026-07-18 11:58:34.307017398 +0000 UTC m=+583.379076900" Jul 18 11:58:45.299195 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:45.299165 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-4kgb4" Jul 18 11:58:51.349400 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:51.349368 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:58:51.349400 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:58:51.349382 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 11:59:19.638769 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.638684 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7"] Jul 18 11:59:19.642386 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.642367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.645298 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.645278 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 11:59:19.645298 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.645293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 18 11:59:19.645482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.645348 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 11:59:19.645482 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.645287 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k9r9z\"" Jul 18 11:59:19.654315 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.654294 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7"] Jul 18 11:59:19.692572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.692572 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.692759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.692759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.692759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ffbk\" (UniqueName: \"kubernetes.io/projected/17ec13aa-16d3-46c8-881e-ee76829abe49-kube-api-access-7ffbk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.692759 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.692715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ec13aa-16d3-46c8-881e-ee76829abe49-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793212 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793327 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ffbk\" (UniqueName: \"kubernetes.io/projected/17ec13aa-16d3-46c8-881e-ee76829abe49-kube-api-access-7ffbk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793421 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ec13aa-16d3-46c8-881e-ee76829abe49-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793674 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793644 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793733 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.793790 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.793757 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.795636 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.795609 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ec13aa-16d3-46c8-881e-ee76829abe49-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.795794 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.795777 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ec13aa-16d3-46c8-881e-ee76829abe49-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.801008 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.800982 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ffbk\" (UniqueName: \"kubernetes.io/projected/17ec13aa-16d3-46c8-881e-ee76829abe49-kube-api-access-7ffbk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c44q7\" (UID: \"17ec13aa-16d3-46c8-881e-ee76829abe49\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:19.954345 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:19.954310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:20.085790 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:20.085766 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7"] Jul 18 11:59:20.087571 ip-10-0-130-242 kubenswrapper[2578]: W0718 11:59:20.087543 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ec13aa_16d3_46c8_881e_ee76829abe49.slice/crio-25ddc80c7b11d5bb0dec9347bf6e1aa44c38a113d7aac8fdf2b1579135fd11f7 WatchSource:0}: Error finding container 25ddc80c7b11d5bb0dec9347bf6e1aa44c38a113d7aac8fdf2b1579135fd11f7: Status 404 returned error can't find the container with id 25ddc80c7b11d5bb0dec9347bf6e1aa44c38a113d7aac8fdf2b1579135fd11f7 Jul 18 11:59:20.088929 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:20.088912 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 11:59:20.530000 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:20.529957 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" event={"ID":"17ec13aa-16d3-46c8-881e-ee76829abe49","Type":"ContainerStarted","Data":"25ddc80c7b11d5bb0dec9347bf6e1aa44c38a113d7aac8fdf2b1579135fd11f7"} Jul 18 11:59:21.537853 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:21.537808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" event={"ID":"17ec13aa-16d3-46c8-881e-ee76829abe49","Type":"ContainerStarted","Data":"acc183d8333190fd5a1bb4dbeffca07ebb471c133dbf8e903c0f71c46178c118"} Jul 18 11:59:21.554544 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:21.554489 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" podStartSLOduration=1.5564203920000002 podStartE2EDuration="2.554469055s" totalImagesPullingTime="998.048663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 11:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 11:59:21.553528013 +0000 UTC m=+630.625587529" watchObservedRunningTime="2026-07-18 11:59:21.554469055 +0000 UTC m=+630.626528560" Jul 18 11:59:29.954968 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:29.954924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:29.955453 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:29.955007 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:29.967536 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:29.967505 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 11:59:30.594939 ip-10-0-130-242 kubenswrapper[2578]: I0718 11:59:30.594906 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c44q7" Jul 18 12:01:17.287327 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.287294 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6844644696-942js"] Jul 18 12:01:17.290573 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.290555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.293902 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.293883 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xlzqh\"" Jul 18 12:01:17.294012 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.293911 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 18 12:01:17.303306 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.303286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6844644696-942js"] Jul 18 12:01:17.387952 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.387914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/235fa772-bfa8-47f4-be66-eb551e598cd7-tls-cert\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.388132 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.388059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vm9d\" (UniqueName: \"kubernetes.io/projected/235fa772-bfa8-47f4-be66-eb551e598cd7-kube-api-access-5vm9d\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.488620 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.488589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vm9d\" (UniqueName: \"kubernetes.io/projected/235fa772-bfa8-47f4-be66-eb551e598cd7-kube-api-access-5vm9d\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.488783 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.488642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/235fa772-bfa8-47f4-be66-eb551e598cd7-tls-cert\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.491041 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.491023 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/235fa772-bfa8-47f4-be66-eb551e598cd7-tls-cert\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.496699 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.496680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vm9d\" (UniqueName: \"kubernetes.io/projected/235fa772-bfa8-47f4-be66-eb551e598cd7-kube-api-access-5vm9d\") pod \"authorino-6844644696-942js\" (UID: \"235fa772-bfa8-47f4-be66-eb551e598cd7\") " pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.598929 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.598839 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6844644696-942js" Jul 18 12:01:17.735714 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:17.735685 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6844644696-942js"] Jul 18 12:01:17.737410 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:01:17.737384 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod235fa772_bfa8_47f4_be66_eb551e598cd7.slice/crio-8ac3f34ba9a7c26dd40aca0e3138f9ea19463d066987a30b9a8115c2c19e9088 WatchSource:0}: Error finding container 8ac3f34ba9a7c26dd40aca0e3138f9ea19463d066987a30b9a8115c2c19e9088: Status 404 returned error can't find the container with id 8ac3f34ba9a7c26dd40aca0e3138f9ea19463d066987a30b9a8115c2c19e9088 Jul 18 12:01:18.128398 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:18.128355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6844644696-942js" event={"ID":"235fa772-bfa8-47f4-be66-eb551e598cd7","Type":"ContainerStarted","Data":"8ac3f34ba9a7c26dd40aca0e3138f9ea19463d066987a30b9a8115c2c19e9088"} Jul 18 12:01:19.135891 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:19.135847 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6844644696-942js" event={"ID":"235fa772-bfa8-47f4-be66-eb551e598cd7","Type":"ContainerStarted","Data":"ba51bb389325bf4c76acba4a86098fb1d460da5c9e993e34cc6c7a000b4bcc8f"} Jul 18 12:01:19.156637 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:01:19.156583 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6844644696-942js" podStartSLOduration=1.702710291 podStartE2EDuration="2.156566032s" totalImagesPullingTime="453.855741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:01:19.154743448 +0000 UTC m=+748.226802952" watchObservedRunningTime="2026-07-18 12:01:19.156566032 +0000 UTC m=+748.228625535" Jul 18 12:03:51.381638 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:03:51.381550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:03:51.382997 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:03:51.382977 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:08:06.058709 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:06.058673 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 12:08:06.059218 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:06.058870 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" podUID="4c462e89-cc83-485e-a0b6-5b46f9855b59" containerName="manager" containerID="cri-o://28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2" gracePeriod=10 Jul 18 12:08:09.807158 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.807125 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 12:08:09.861400 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.861359 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume\") pod \"4c462e89-cc83-485e-a0b6-5b46f9855b59\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " Jul 18 12:08:09.861567 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.861430 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxb7l\" (UniqueName: \"kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l\") pod \"4c462e89-cc83-485e-a0b6-5b46f9855b59\" (UID: \"4c462e89-cc83-485e-a0b6-5b46f9855b59\") " Jul 18 12:08:09.861731 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.861701 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4c462e89-cc83-485e-a0b6-5b46f9855b59" (UID: "4c462e89-cc83-485e-a0b6-5b46f9855b59"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:08:09.863434 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.863406 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l" (OuterVolumeSpecName: "kube-api-access-vxb7l") pod "4c462e89-cc83-485e-a0b6-5b46f9855b59" (UID: "4c462e89-cc83-485e-a0b6-5b46f9855b59"). InnerVolumeSpecName "kube-api-access-vxb7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:08:09.963136 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.963107 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c462e89-cc83-485e-a0b6-5b46f9855b59-extensions-socket-volume\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:08:09.963136 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:09.963132 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxb7l\" (UniqueName: \"kubernetes.io/projected/4c462e89-cc83-485e-a0b6-5b46f9855b59-kube-api-access-vxb7l\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:08:10.299293 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.299256 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c462e89-cc83-485e-a0b6-5b46f9855b59" containerID="28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2" exitCode=0 Jul 18 12:08:10.299763 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.299300 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" Jul 18 12:08:10.299763 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.299334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" event={"ID":"4c462e89-cc83-485e-a0b6-5b46f9855b59","Type":"ContainerDied","Data":"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2"} Jul 18 12:08:10.299763 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.299382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9" event={"ID":"4c462e89-cc83-485e-a0b6-5b46f9855b59","Type":"ContainerDied","Data":"f9c399c366df934ab712d02ba19fd8b68599488c9898eae3703afdb1a5b41442"} Jul 18 12:08:10.299763 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.299407 2578 scope.go:117] "RemoveContainer" containerID="28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2" Jul 18 12:08:10.309874 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.309855 2578 scope.go:117] "RemoveContainer" containerID="28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2" Jul 18 12:08:10.310154 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:08:10.310135 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2\": container with ID starting with 28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2 not found: ID does not exist" containerID="28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2" Jul 18 12:08:10.310222 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.310162 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2"} err="failed to get container status \"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2\": rpc error: code = NotFound desc = could not find container \"28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2\": container with ID starting with 28652773d95131418527ddcb31f316e06a383c159a80967bc3bafd88badf94e2 not found: ID does not exist" Jul 18 12:08:10.321747 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.321720 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 12:08:10.327724 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:10.327698 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l85g9"] Jul 18 12:08:11.412925 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:11.412892 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c462e89-cc83-485e-a0b6-5b46f9855b59" path="/var/lib/kubelet/pods/4c462e89-cc83-485e-a0b6-5b46f9855b59/volumes" Jul 18 12:08:51.415285 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:51.415257 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:08:51.421628 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:08:51.421597 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:09:07.142087 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.142041 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89"] Jul 18 12:09:07.142537 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.142383 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c462e89-cc83-485e-a0b6-5b46f9855b59" containerName="manager" Jul 18 12:09:07.142537 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.142394 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c462e89-cc83-485e-a0b6-5b46f9855b59" containerName="manager" Jul 18 12:09:07.142537 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.142470 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c462e89-cc83-485e-a0b6-5b46f9855b59" containerName="manager" Jul 18 12:09:07.146648 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.146630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.149241 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.149222 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qvmch\"" Jul 18 12:09:07.159764 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.159734 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89"] Jul 18 12:09:07.240581 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.240520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2vf2\" (UniqueName: \"kubernetes.io/projected/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-kube-api-access-v2vf2\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.240780 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.240684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.341124 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.341085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.341124 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.341129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2vf2\" (UniqueName: \"kubernetes.io/projected/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-kube-api-access-v2vf2\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.341525 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.341504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.350090 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.350065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2vf2\" (UniqueName: \"kubernetes.io/projected/3d46cbd9-8e7a-43da-a619-94ab8df8ad3e-kube-api-access-v2vf2\") pod \"kuadrant-operator-controller-manager-86878c769f-whl89\" (UID: \"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.456957 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.456917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:07.595018 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:07.594997 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89"] Jul 18 12:09:07.597008 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:09:07.596980 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d46cbd9_8e7a_43da_a619_94ab8df8ad3e.slice/crio-ca971416e3561c39f8c59cba8b76a526516d2be94bdde0247595b4ba5194b5cd WatchSource:0}: Error finding container ca971416e3561c39f8c59cba8b76a526516d2be94bdde0247595b4ba5194b5cd: Status 404 returned error can't find the container with id ca971416e3561c39f8c59cba8b76a526516d2be94bdde0247595b4ba5194b5cd Jul 18 12:09:08.594585 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:08.594549 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" event={"ID":"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e","Type":"ContainerStarted","Data":"0e7b23a1044b332a560662e59302c092a36889ecb119a9e1a954fa619c2a3a22"} Jul 18 12:09:08.594987 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:08.594602 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" event={"ID":"3d46cbd9-8e7a-43da-a619-94ab8df8ad3e","Type":"ContainerStarted","Data":"ca971416e3561c39f8c59cba8b76a526516d2be94bdde0247595b4ba5194b5cd"} Jul 18 12:09:08.594987 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:08.594687 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:09:08.614942 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:08.614897 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" podStartSLOduration=1.614881644 podStartE2EDuration="1.614881644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:09:08.612426211 +0000 UTC m=+1217.684485715" watchObservedRunningTime="2026-07-18 12:09:08.614881644 +0000 UTC m=+1217.686941148" Jul 18 12:09:19.602354 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:09:19.602322 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-whl89" Jul 18 12:13:51.456134 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:13:51.456101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:13:51.458537 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:13:51.458516 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:18:51.488305 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:18:51.488269 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:18:51.492823 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:18:51.492800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:20:20.763762 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.763680 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z"] Jul 18 12:20:20.767065 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.767033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:20.769594 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.769568 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:20:20.769594 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.769588 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:20:20.774864 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.774841 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z"] Jul 18 12:20:20.894380 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.894350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z\" (UID: \"b6116214-a781-4309-ba43-adf83c9e7a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:20.995840 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.995800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z\" (UID: \"b6116214-a781-4309-ba43-adf83c9e7a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:20.996587 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:20.996568 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z\" (UID: \"b6116214-a781-4309-ba43-adf83c9e7a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:21.079191 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:21.079118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:21.210941 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:21.210907 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z"] Jul 18 12:20:21.212437 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:20:21.212401 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6116214_a781_4309_ba43_adf83c9e7a4d.slice/crio-5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb WatchSource:0}: Error finding container 5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb: Status 404 returned error can't find the container with id 5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb Jul 18 12:20:21.214464 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:21.214447 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 12:20:22.077515 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:22.077471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" event={"ID":"b6116214-a781-4309-ba43-adf83c9e7a4d","Type":"ContainerStarted","Data":"5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb"} Jul 18 12:20:24.089747 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:24.089710 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" event={"ID":"b6116214-a781-4309-ba43-adf83c9e7a4d","Type":"ContainerStarted","Data":"e66a074f6fa5219c40059c3795ece68b6a4e49e21ec8d1e0310eadba58b08c5b"} Jul 18 12:20:24.110697 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:24.110646 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" podStartSLOduration=1.864957153 podStartE2EDuration="4.110627671s" totalImagesPullingTime="2.245670518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:20:24.107378441 +0000 UTC m=+1893.179437944" watchObservedRunningTime="2026-07-18 12:20:24.110627671 +0000 UTC m=+1893.182687186" Jul 18 12:20:25.095834 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:25.095798 2578 generic.go:356] "Generic (PLEG): container finished" podID="b6116214-a781-4309-ba43-adf83c9e7a4d" containerID="e66a074f6fa5219c40059c3795ece68b6a4e49e21ec8d1e0310eadba58b08c5b" exitCode=0 Jul 18 12:20:25.096297 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:25.095877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" event={"ID":"b6116214-a781-4309-ba43-adf83c9e7a4d","Type":"ContainerDied","Data":"e66a074f6fa5219c40059c3795ece68b6a4e49e21ec8d1e0310eadba58b08c5b"} Jul 18 12:20:26.229967 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:26.229938 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:26.346508 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:26.346477 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca\") pod \"b6116214-a781-4309-ba43-adf83c9e7a4d\" (UID: \"b6116214-a781-4309-ba43-adf83c9e7a4d\") " Jul 18 12:20:26.346820 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:26.346796 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6116214-a781-4309-ba43-adf83c9e7a4d" (UID: "b6116214-a781-4309-ba43-adf83c9e7a4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:20:26.447797 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:26.447763 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6116214-a781-4309-ba43-adf83c9e7a4d-maas-api-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:20:27.107652 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:27.107624 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" Jul 18 12:20:27.107819 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:27.107666 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z" event={"ID":"b6116214-a781-4309-ba43-adf83c9e7a4d","Type":"ContainerDied","Data":"5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb"} Jul 18 12:20:27.107819 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:27.107709 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b1e7457dd1ad5f0247666ff9756e7eb4094e35f927e24fa586832edf9d6ebdb" Jul 18 12:20:35.850720 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:35.850686 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z"] Jul 18 12:20:35.853618 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:35.853592 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3dff9e6d-ns85z"] Jul 18 12:20:37.411979 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:20:37.411940 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6116214-a781-4309-ba43-adf83c9e7a4d" path="/var/lib/kubelet/pods/b6116214-a781-4309-ba43-adf83c9e7a4d/volumes" Jul 18 12:23:19.474130 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.474093 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr"] Jul 18 12:23:19.474558 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.474449 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6116214-a781-4309-ba43-adf83c9e7a4d" containerName="revoke-keys" Jul 18 12:23:19.474558 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.474460 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6116214-a781-4309-ba43-adf83c9e7a4d" containerName="revoke-keys" Jul 18 12:23:19.474558 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.474536 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6116214-a781-4309-ba43-adf83c9e7a4d" containerName="revoke-keys" Jul 18 12:23:19.477449 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.477430 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:19.480017 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.479993 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:23:19.480142 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.480004 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:23:19.486899 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.486875 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr"] Jul 18 12:23:19.603152 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.603119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr\" (UID: \"7d543242-da35-4d40-9002-a0e69fbc1ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:19.704615 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.704581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr\" (UID: \"7d543242-da35-4d40-9002-a0e69fbc1ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:19.705253 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.705231 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr\" (UID: \"7d543242-da35-4d40-9002-a0e69fbc1ae6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:19.788622 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:19.788535 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:20.136612 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:20.136479 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr"] Jul 18 12:23:20.139193 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:23:20.139165 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d543242_da35_4d40_9002_a0e69fbc1ae6.slice/crio-20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d WatchSource:0}: Error finding container 20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d: Status 404 returned error can't find the container with id 20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d Jul 18 12:23:21.031573 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:21.031535 2578 generic.go:356] "Generic (PLEG): container finished" podID="7d543242-da35-4d40-9002-a0e69fbc1ae6" containerID="c90971b2e79bf0e92ad643fa20a61617a01fc03622aed22374c50b122dffd856" exitCode=0 Jul 18 12:23:21.031963 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:21.031607 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" event={"ID":"7d543242-da35-4d40-9002-a0e69fbc1ae6","Type":"ContainerDied","Data":"c90971b2e79bf0e92ad643fa20a61617a01fc03622aed22374c50b122dffd856"} Jul 18 12:23:21.031963 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:21.031653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" event={"ID":"7d543242-da35-4d40-9002-a0e69fbc1ae6","Type":"ContainerStarted","Data":"20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d"} Jul 18 12:23:22.174823 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:22.174797 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:22.224814 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:22.224775 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca\") pod \"7d543242-da35-4d40-9002-a0e69fbc1ae6\" (UID: \"7d543242-da35-4d40-9002-a0e69fbc1ae6\") " Jul 18 12:23:22.225184 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:22.225161 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7d543242-da35-4d40-9002-a0e69fbc1ae6" (UID: "7d543242-da35-4d40-9002-a0e69fbc1ae6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:23:22.326302 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:22.326211 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d543242-da35-4d40-9002-a0e69fbc1ae6-maas-api-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:23:23.042535 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:23.042510 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" Jul 18 12:23:23.042706 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:23.042504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr" event={"ID":"7d543242-da35-4d40-9002-a0e69fbc1ae6","Type":"ContainerDied","Data":"20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d"} Jul 18 12:23:23.042706 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:23.042639 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20e672310f0381c4f6b5555dadd9c0b6d6bc4a2a5d6a0f2328a4ee26dceadb5d" Jul 18 12:23:34.560634 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:34.560597 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr"] Jul 18 12:23:34.562729 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:34.562704 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a9c69c8d-j5mtr"] Jul 18 12:23:35.411960 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:35.411924 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d543242-da35-4d40-9002-a0e69fbc1ae6" path="/var/lib/kubelet/pods/7d543242-da35-4d40-9002-a0e69fbc1ae6/volumes" Jul 18 12:23:51.523172 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:51.523145 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:23:51.528738 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:23:51.528715 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:24:47.665971 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.665890 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2"] Jul 18 12:24:47.666469 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.666238 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d543242-da35-4d40-9002-a0e69fbc1ae6" containerName="revoke-keys" Jul 18 12:24:47.666469 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.666249 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d543242-da35-4d40-9002-a0e69fbc1ae6" containerName="revoke-keys" Jul 18 12:24:47.666469 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.666317 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d543242-da35-4d40-9002-a0e69fbc1ae6" containerName="revoke-keys" Jul 18 12:24:47.669254 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.669239 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:47.671739 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.671716 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:24:47.671888 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.671871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:24:47.679787 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.679763 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2"] Jul 18 12:24:47.841714 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.841681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2\" (UID: \"ba13e30f-0b74-4cf0-82bf-b29e05db86bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:47.942319 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.942293 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2\" (UID: \"ba13e30f-0b74-4cf0-82bf-b29e05db86bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:47.942909 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.942891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2\" (UID: \"ba13e30f-0b74-4cf0-82bf-b29e05db86bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:47.980337 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:47.980310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:48.113191 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:48.113160 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2"] Jul 18 12:24:48.114782 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:24:48.114754 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba13e30f_0b74_4cf0_82bf_b29e05db86bd.slice/crio-9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4 WatchSource:0}: Error finding container 9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4: Status 404 returned error can't find the container with id 9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4 Jul 18 12:24:48.492065 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:48.492011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" event={"ID":"ba13e30f-0b74-4cf0-82bf-b29e05db86bd","Type":"ContainerStarted","Data":"70e854b977eac25a78c396474a04e9e7da7d7afabc7d72e4cbdec40540cd40d2"} Jul 18 12:24:48.492263 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:48.492093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" event={"ID":"ba13e30f-0b74-4cf0-82bf-b29e05db86bd","Type":"ContainerStarted","Data":"9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4"} Jul 18 12:24:48.508920 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:48.508873 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" podStartSLOduration=1.508859237 podStartE2EDuration="1.508859237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:24:48.508030723 +0000 UTC m=+2157.580090226" watchObservedRunningTime="2026-07-18 12:24:48.508859237 +0000 UTC m=+2157.580918740" Jul 18 12:24:49.498056 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:49.498007 2578 generic.go:356] "Generic (PLEG): container finished" podID="ba13e30f-0b74-4cf0-82bf-b29e05db86bd" containerID="70e854b977eac25a78c396474a04e9e7da7d7afabc7d72e4cbdec40540cd40d2" exitCode=0 Jul 18 12:24:49.498443 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:49.498088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" event={"ID":"ba13e30f-0b74-4cf0-82bf-b29e05db86bd","Type":"ContainerDied","Data":"70e854b977eac25a78c396474a04e9e7da7d7afabc7d72e4cbdec40540cd40d2"} Jul 18 12:24:50.633660 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:50.633636 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:50.769613 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:50.769517 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca\") pod \"ba13e30f-0b74-4cf0-82bf-b29e05db86bd\" (UID: \"ba13e30f-0b74-4cf0-82bf-b29e05db86bd\") " Jul 18 12:24:50.769924 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:50.769900 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba13e30f-0b74-4cf0-82bf-b29e05db86bd" (UID: "ba13e30f-0b74-4cf0-82bf-b29e05db86bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:24:50.870633 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:50.870604 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba13e30f-0b74-4cf0-82bf-b29e05db86bd-maas-api-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:24:51.508530 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:51.508499 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" Jul 18 12:24:51.508693 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:51.508504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2" event={"ID":"ba13e30f-0b74-4cf0-82bf-b29e05db86bd","Type":"ContainerDied","Data":"9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4"} Jul 18 12:24:51.508693 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:24:51.508631 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a03661cd03aa4be537cd054890d49e3bc4ed818de3cb8b1caaf51d4e3af64e4" Jul 18 12:25:07.740410 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:25:07.740369 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2"] Jul 18 12:25:07.743137 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:25:07.743111 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-07bfbcac-7t8c2"] Jul 18 12:25:09.411822 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:25:09.411789 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba13e30f-0b74-4cf0-82bf-b29e05db86bd" path="/var/lib/kubelet/pods/ba13e30f-0b74-4cf0-82bf-b29e05db86bd/volumes" Jul 18 12:26:51.601736 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:26:51.601700 2578 scope.go:117] "RemoveContainer" containerID="e66a074f6fa5219c40059c3795ece68b6a4e49e21ec8d1e0310eadba58b08c5b" Jul 18 12:28:23.294787 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.294751 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t"] Jul 18 12:28:23.295242 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.295138 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba13e30f-0b74-4cf0-82bf-b29e05db86bd" containerName="revoke-keys" Jul 18 12:28:23.295242 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.295153 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba13e30f-0b74-4cf0-82bf-b29e05db86bd" containerName="revoke-keys" Jul 18 12:28:23.295242 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.295227 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba13e30f-0b74-4cf0-82bf-b29e05db86bd" containerName="revoke-keys" Jul 18 12:28:23.298196 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.298179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:23.300775 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.300746 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:28:23.300899 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.300810 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:28:23.309992 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.309970 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t"] Jul 18 12:28:23.363816 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.363781 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t\" (UID: \"b8630a68-05fc-4cc1-ad55-bd8f932e9603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:23.465200 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.465163 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t\" (UID: \"b8630a68-05fc-4cc1-ad55-bd8f932e9603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:23.465743 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.465725 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t\" (UID: \"b8630a68-05fc-4cc1-ad55-bd8f932e9603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:23.608776 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.608691 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:23.963946 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:23.963920 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t"] Jul 18 12:28:23.965447 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:28:23.965418 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8630a68_05fc_4cc1_ad55_bd8f932e9603.slice/crio-ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e WatchSource:0}: Error finding container ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e: Status 404 returned error can't find the container with id ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e Jul 18 12:28:24.660192 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:24.660088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" event={"ID":"b8630a68-05fc-4cc1-ad55-bd8f932e9603","Type":"ContainerStarted","Data":"c87115afaf777700b4a97704acb96e95d57b735a96707e1e67576322449ad44b"} Jul 18 12:28:24.660192 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:24.660180 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" event={"ID":"b8630a68-05fc-4cc1-ad55-bd8f932e9603","Type":"ContainerStarted","Data":"ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e"} Jul 18 12:28:24.680613 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:24.680565 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" podStartSLOduration=1.680551299 podStartE2EDuration="1.680551299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:28:24.677878674 +0000 UTC m=+2373.749938177" watchObservedRunningTime="2026-07-18 12:28:24.680551299 +0000 UTC m=+2373.752610844" Jul 18 12:28:25.666656 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:25.666623 2578 generic.go:356] "Generic (PLEG): container finished" podID="b8630a68-05fc-4cc1-ad55-bd8f932e9603" containerID="c87115afaf777700b4a97704acb96e95d57b735a96707e1e67576322449ad44b" exitCode=0 Jul 18 12:28:25.667091 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:25.666707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" event={"ID":"b8630a68-05fc-4cc1-ad55-bd8f932e9603","Type":"ContainerDied","Data":"c87115afaf777700b4a97704acb96e95d57b735a96707e1e67576322449ad44b"} Jul 18 12:28:26.802292 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:26.802262 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:26.895647 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:26.895608 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca\") pod \"b8630a68-05fc-4cc1-ad55-bd8f932e9603\" (UID: \"b8630a68-05fc-4cc1-ad55-bd8f932e9603\") " Jul 18 12:28:26.895956 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:26.895933 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b8630a68-05fc-4cc1-ad55-bd8f932e9603" (UID: "b8630a68-05fc-4cc1-ad55-bd8f932e9603"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:28:26.996819 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:26.996738 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8630a68-05fc-4cc1-ad55-bd8f932e9603-maas-api-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:28:27.677842 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:27.677754 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" Jul 18 12:28:27.677842 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:27.677764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t" event={"ID":"b8630a68-05fc-4cc1-ad55-bd8f932e9603","Type":"ContainerDied","Data":"ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e"} Jul 18 12:28:27.677842 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:27.677819 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee51ceb7c794fa9b6eb75ad4a85ed6c40dccd538dadbd1c81d6af213beade91e" Jul 18 12:28:38.454565 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:38.454528 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t"] Jul 18 12:28:38.457777 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:38.457750 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3be5a3-9z89t"] Jul 18 12:28:39.411900 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:39.411856 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8630a68-05fc-4cc1-ad55-bd8f932e9603" path="/var/lib/kubelet/pods/b8630a68-05fc-4cc1-ad55-bd8f932e9603/volumes" Jul 18 12:28:51.557205 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:51.557178 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:28:51.563475 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:28:51.563452 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:29:51.640636 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:29:51.640607 2578 scope.go:117] "RemoveContainer" containerID="c90971b2e79bf0e92ad643fa20a61617a01fc03622aed22374c50b122dffd856" Jul 18 12:30:00.138949 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.138911 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n"] Jul 18 12:30:00.139484 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.139465 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8630a68-05fc-4cc1-ad55-bd8f932e9603" containerName="revoke-keys" Jul 18 12:30:00.139563 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.139488 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8630a68-05fc-4cc1-ad55-bd8f932e9603" containerName="revoke-keys" Jul 18 12:30:00.139646 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.139633 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8630a68-05fc-4cc1-ad55-bd8f932e9603" containerName="revoke-keys" Jul 18 12:30:00.142671 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.142651 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h"] Jul 18 12:30:00.142800 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.142778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" Jul 18 12:30:00.145341 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.145320 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:30:00.145939 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.145916 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" Jul 18 12:30:00.149659 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.149632 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n"] Jul 18 12:30:00.153581 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.153558 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h"] Jul 18 12:30:00.156590 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.156572 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" Jul 18 12:30:00.168112 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.168085 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" Jul 18 12:30:00.303829 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.303799 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n"] Jul 18 12:30:00.332516 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:00.332485 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h"] Jul 18 12:30:00.333808 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:30:00.333780 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8c51fe_badf_40b9_8cc1_0167ecc9747d.slice/crio-bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7 WatchSource:0}: Error finding container bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7: Status 404 returned error can't find the container with id bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7 Jul 18 12:30:01.178877 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.178842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" event={"ID":"c7cede72-3354-4f47-ba86-729ec3e4137d","Type":"ContainerStarted","Data":"10d6dc6cd5fc6e09bfe0cc714f9223bf04cac2303171e0435c2d7a69857f99db"} Jul 18 12:30:01.179463 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.178893 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" event={"ID":"c7cede72-3354-4f47-ba86-729ec3e4137d","Type":"ContainerStarted","Data":"ea81a1c23afb877cac1e41b0f1e01c0cc8acc1ab44dc95b8d26f375a0d7f90e3"} Jul 18 12:30:01.180345 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.180320 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" event={"ID":"bf8c51fe-badf-40b9-8cc1-0167ecc9747d","Type":"ContainerStarted","Data":"2ea73eb797952bae7c28733bdc230d4b3b63597ac3134eb2451ed35b3d74d192"} Jul 18 12:30:01.180433 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.180361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" event={"ID":"bf8c51fe-badf-40b9-8cc1-0167ecc9747d","Type":"ContainerStarted","Data":"bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7"} Jul 18 12:30:01.194782 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.194742 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" podStartSLOduration=1.194728659 podStartE2EDuration="1.194728659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:30:01.193581819 +0000 UTC m=+2470.265641304" watchObservedRunningTime="2026-07-18 12:30:01.194728659 +0000 UTC m=+2470.266788221" Jul 18 12:30:01.211739 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:01.211685 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" podStartSLOduration=1.211667557 podStartE2EDuration="1.211667557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:30:01.20934566 +0000 UTC m=+2470.281405164" watchObservedRunningTime="2026-07-18 12:30:01.211667557 +0000 UTC m=+2470.283727060" Jul 18 12:30:02.187091 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:02.187038 2578 generic.go:356] "Generic (PLEG): container finished" podID="c7cede72-3354-4f47-ba86-729ec3e4137d" containerID="10d6dc6cd5fc6e09bfe0cc714f9223bf04cac2303171e0435c2d7a69857f99db" exitCode=0 Jul 18 12:30:02.187516 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:02.187137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" event={"ID":"c7cede72-3354-4f47-ba86-729ec3e4137d","Type":"ContainerDied","Data":"10d6dc6cd5fc6e09bfe0cc714f9223bf04cac2303171e0435c2d7a69857f99db"} Jul 18 12:30:02.188626 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:02.188604 2578 generic.go:356] "Generic (PLEG): container finished" podID="bf8c51fe-badf-40b9-8cc1-0167ecc9747d" containerID="2ea73eb797952bae7c28733bdc230d4b3b63597ac3134eb2451ed35b3d74d192" exitCode=0 Jul 18 12:30:02.188716 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:02.188649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" event={"ID":"bf8c51fe-badf-40b9-8cc1-0167ecc9747d","Type":"ContainerDied","Data":"2ea73eb797952bae7c28733bdc230d4b3b63597ac3134eb2451ed35b3d74d192"} Jul 18 12:30:03.321452 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:03.321430 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" Jul 18 12:30:03.363206 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:03.363180 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" Jul 18 12:30:04.201151 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.201113 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" Jul 18 12:30:04.201151 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.201129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n" event={"ID":"c7cede72-3354-4f47-ba86-729ec3e4137d","Type":"ContainerDied","Data":"ea81a1c23afb877cac1e41b0f1e01c0cc8acc1ab44dc95b8d26f375a0d7f90e3"} Jul 18 12:30:04.201432 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.201205 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea81a1c23afb877cac1e41b0f1e01c0cc8acc1ab44dc95b8d26f375a0d7f90e3" Jul 18 12:30:04.202466 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.202448 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" Jul 18 12:30:04.202581 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.202495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739630-t2k6h" event={"ID":"bf8c51fe-badf-40b9-8cc1-0167ecc9747d","Type":"ContainerDied","Data":"bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7"} Jul 18 12:30:04.202581 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:04.202532 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc28b43d3624b67024d6386fbe71eb6bbf04b9eb141bad17b6c61f43f8fe8fe7" Jul 18 12:30:14.553587 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:14.553556 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n"] Jul 18 12:30:14.557115 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:14.557091 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b7127d-29739630-b8p6n"] Jul 18 12:30:15.411740 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:15.411704 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7cede72-3354-4f47-ba86-729ec3e4137d" path="/var/lib/kubelet/pods/c7cede72-3354-4f47-ba86-729ec3e4137d/volumes" Jul 18 12:30:51.660828 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:30:51.660751 2578 scope.go:117] "RemoveContainer" containerID="70e854b977eac25a78c396474a04e9e7da7d7afabc7d72e4cbdec40540cd40d2" Jul 18 12:31:26.935554 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935524 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935855 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8c51fe-badf-40b9-8cc1-0167ecc9747d" containerName="cleanup" Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935866 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8c51fe-badf-40b9-8cc1-0167ecc9747d" containerName="cleanup" Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935884 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cede72-3354-4f47-ba86-729ec3e4137d" containerName="cleanup" Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935890 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cede72-3354-4f47-ba86-729ec3e4137d" containerName="cleanup" Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935944 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cede72-3354-4f47-ba86-729ec3e4137d" containerName="cleanup" Jul 18 12:31:26.937533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.935960 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8c51fe-badf-40b9-8cc1-0167ecc9747d" containerName="cleanup" Jul 18 12:31:26.938756 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.938737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:26.941551 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.941531 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"kube-root-ca.crt\"" Jul 18 12:31:26.942474 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.942448 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"openshift-service-ca.crt\"" Jul 18 12:31:26.942575 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.942485 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"auth-test-model-8b9f97-kserve-self-signed-certs\"" Jul 18 12:31:26.942575 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.942550 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"default-dockercfg-hq56s\"" Jul 18 12:31:26.948997 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:26.948974 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:31:27.030747 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.030747 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030745 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.030986 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.030986 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.030986 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.030986 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.030927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77nn\" (UniqueName: \"kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132370 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132370 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132614 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77nn\" (UniqueName: \"kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132614 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132614 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132614 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132828 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132760 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.132892 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.132850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.133175 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.133154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.135395 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.135370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.135756 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.135737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.141423 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.141401 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77nn\" (UniqueName: \"kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn\") pod \"auth-test-model-8b9f97-kserve-776d54c59f-5p2st\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.249263 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.249182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:27.384569 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.384538 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:31:27.645955 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.645861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" event={"ID":"c888f93b-1d6c-4689-b011-b9114a11c5aa","Type":"ContainerStarted","Data":"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002"} Jul 18 12:31:27.645955 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.645906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" event={"ID":"c888f93b-1d6c-4689-b011-b9114a11c5aa","Type":"ContainerStarted","Data":"94eb8a598b8a2475f2c28102edc0a36cb2c59671b54b341d65b961cedb26de80"} Jul 18 12:31:27.666410 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:27.666361 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" podStartSLOduration=1.666347097 podStartE2EDuration="1.666347097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:31:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:31:27.664588918 +0000 UTC m=+2556.736648432" watchObservedRunningTime="2026-07-18 12:31:27.666347097 +0000 UTC m=+2556.738406599" Jul 18 12:31:37.250269 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:37.250232 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:37.250654 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:37.250279 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:37.262423 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:37.262399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:31:37.708319 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:31:37.708286 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:33:29.522064 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:29.521999 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:33:44.544442 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.544358 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" podUID="c888f93b-1d6c-4689-b011-b9114a11c5aa" containerName="main" containerID="cri-o://0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002" gracePeriod=45 Jul 18 12:33:44.796732 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.796673 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:33:44.939672 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939639 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d77nn\" (UniqueName: \"kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.939837 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939699 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.939837 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939815 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.939943 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939845 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.939943 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939867 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.939943 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939890 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm\") pod \"c888f93b-1d6c-4689-b011-b9114a11c5aa\" (UID: \"c888f93b-1d6c-4689-b011-b9114a11c5aa\") " Jul 18 12:33:44.940133 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.939937 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home" (OuterVolumeSpecName: "home") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:33:44.940133 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.940097 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:33:44.940240 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.940123 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:33:44.940240 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.940230 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-home\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:44.940333 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.940248 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-tmp-dir\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:44.940333 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.940263 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-model-cache\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:44.941871 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.941844 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn" (OuterVolumeSpecName: "kube-api-access-d77nn") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "kube-api-access-d77nn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:33:44.941980 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.941875 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm" (OuterVolumeSpecName: "dshm") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:33:44.941980 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:44.941952 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c888f93b-1d6c-4689-b011-b9114a11c5aa" (UID: "c888f93b-1d6c-4689-b011-b9114a11c5aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:33:45.041601 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.041564 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c888f93b-1d6c-4689-b011-b9114a11c5aa-tls-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:45.041601 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.041593 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c888f93b-1d6c-4689-b011-b9114a11c5aa-dshm\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:45.041601 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.041603 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d77nn\" (UniqueName: \"kubernetes.io/projected/c888f93b-1d6c-4689-b011-b9114a11c5aa-kube-api-access-d77nn\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:33:45.379875 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.379835 2578 generic.go:356] "Generic (PLEG): container finished" podID="c888f93b-1d6c-4689-b011-b9114a11c5aa" containerID="0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002" exitCode=0 Jul 18 12:33:45.379875 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.379876 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" Jul 18 12:33:45.380110 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.379990 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" event={"ID":"c888f93b-1d6c-4689-b011-b9114a11c5aa","Type":"ContainerDied","Data":"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002"} Jul 18 12:33:45.380110 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.380020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st" event={"ID":"c888f93b-1d6c-4689-b011-b9114a11c5aa","Type":"ContainerDied","Data":"94eb8a598b8a2475f2c28102edc0a36cb2c59671b54b341d65b961cedb26de80"} Jul 18 12:33:45.380110 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.380037 2578 scope.go:117] "RemoveContainer" containerID="0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002" Jul 18 12:33:45.389478 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.389460 2578 scope.go:117] "RemoveContainer" containerID="0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002" Jul 18 12:33:45.389738 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:33:45.389718 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002\": container with ID starting with 0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002 not found: ID does not exist" containerID="0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002" Jul 18 12:33:45.389811 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.389746 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002"} err="failed to get container status \"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002\": rpc error: code = NotFound desc = could not find container \"0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002\": container with ID starting with 0bc2c7f28c8997081f74d8af971b01de7f022adea4bf2990f8d812baf8cf6002 not found: ID does not exist" Jul 18 12:33:45.401969 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.401943 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:33:45.406244 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.406220 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/auth-test-model-8b9f97-kserve-776d54c59f-5p2st"] Jul 18 12:33:45.411817 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:45.411795 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c888f93b-1d6c-4689-b011-b9114a11c5aa" path="/var/lib/kubelet/pods/c888f93b-1d6c-4689-b011-b9114a11c5aa/volumes" Jul 18 12:33:51.591832 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:51.591803 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:33:51.599346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:33:51.599327 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tqdw9_85bdf996-67e4-41f3-95fb-798cc6c89502/ovn-acl-logging/0.log" Jul 18 12:34:15.555901 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.555868 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:34:15.556491 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.556339 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c888f93b-1d6c-4689-b011-b9114a11c5aa" containerName="main" Jul 18 12:34:15.556491 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.556352 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c888f93b-1d6c-4689-b011-b9114a11c5aa" containerName="main" Jul 18 12:34:15.556491 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.556413 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c888f93b-1d6c-4689-b011-b9114a11c5aa" containerName="main" Jul 18 12:34:15.559513 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.559498 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.563039 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.563014 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"sub-test-model-8b9f97-kserve-self-signed-certs\"" Jul 18 12:34:15.563039 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.563033 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"openshift-service-ca.crt\"" Jul 18 12:34:15.563267 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.563064 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"kube-root-ca.crt\"" Jul 18 12:34:15.563267 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.563170 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"default-dockercfg-hq56s\"" Jul 18 12:34:15.570897 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.570877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:34:15.683250 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.683250 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.683468 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.683468 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.683468 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkz4g\" (UniqueName: \"kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.683468 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.683379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784161 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784346 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkz4g\" (UniqueName: \"kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784633 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784633 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.784714 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.784689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.786556 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.786539 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.786750 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.786734 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.793581 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.793561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkz4g\" (UniqueName: \"kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g\") pod \"sub-test-model-8b9f97-kserve-75557cfd77-j75nj\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:15.869324 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:15.869255 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:16.009007 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:16.008979 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:34:16.010929 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:34:16.010903 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod481e5c09_af0e_49ca_9ca8_f9d12958bb69.slice/crio-874f6dd4fa26525d81b01d1474548bbdcec3908457a0559c5e114a04ebea990f WatchSource:0}: Error finding container 874f6dd4fa26525d81b01d1474548bbdcec3908457a0559c5e114a04ebea990f: Status 404 returned error can't find the container with id 874f6dd4fa26525d81b01d1474548bbdcec3908457a0559c5e114a04ebea990f Jul 18 12:34:16.551259 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:16.551227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" event={"ID":"481e5c09-af0e-49ca-9ca8-f9d12958bb69","Type":"ContainerStarted","Data":"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9"} Jul 18 12:34:16.551259 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:16.551267 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" event={"ID":"481e5c09-af0e-49ca-9ca8-f9d12958bb69","Type":"ContainerStarted","Data":"874f6dd4fa26525d81b01d1474548bbdcec3908457a0559c5e114a04ebea990f"} Jul 18 12:34:16.567621 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:16.567574 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" podStartSLOduration=1.567560572 podStartE2EDuration="1.567560572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:34:16.566291384 +0000 UTC m=+2725.638350887" watchObservedRunningTime="2026-07-18 12:34:16.567560572 +0000 UTC m=+2725.639620075" Jul 18 12:34:25.870261 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:25.870210 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:25.870261 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:25.870263 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:25.883193 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:25.883164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:26.619443 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:26.619405 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:34:51.709996 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:34:51.709957 2578 scope.go:117] "RemoveContainer" containerID="c87115afaf777700b4a97704acb96e95d57b735a96707e1e67576322449ad44b" Jul 18 12:35:05.456595 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:05.456557 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:35:20.477123 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.476981 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" podUID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" containerName="main" containerID="cri-o://3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9" gracePeriod=45 Jul 18 12:35:20.733446 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.733372 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:35:20.847871 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.847838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.847871 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.847877 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.848144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.847917 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.848144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.847947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.848144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.847993 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.848144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.848021 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkz4g\" (UniqueName: \"kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g\") pod \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\" (UID: \"481e5c09-af0e-49ca-9ca8-f9d12958bb69\") " Jul 18 12:35:20.848144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.848109 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache" (OuterVolumeSpecName: "model-cache") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:35:20.848401 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.848262 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home" (OuterVolumeSpecName: "home") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:35:20.848401 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.848304 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-model-cache\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:20.848401 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.848318 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:35:20.850127 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.850096 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm" (OuterVolumeSpecName: "dshm") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:35:20.850226 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.850191 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g" (OuterVolumeSpecName: "kube-api-access-jkz4g") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "kube-api-access-jkz4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:35:20.850226 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.850188 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "481e5c09-af0e-49ca-9ca8-f9d12958bb69" (UID: "481e5c09-af0e-49ca-9ca8-f9d12958bb69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:35:20.884925 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.884901 2578 generic.go:356] "Generic (PLEG): container finished" podID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" containerID="3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9" exitCode=0 Jul 18 12:35:20.885012 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.884941 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" Jul 18 12:35:20.885012 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.884993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" event={"ID":"481e5c09-af0e-49ca-9ca8-f9d12958bb69","Type":"ContainerDied","Data":"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9"} Jul 18 12:35:20.885126 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.885040 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj" event={"ID":"481e5c09-af0e-49ca-9ca8-f9d12958bb69","Type":"ContainerDied","Data":"874f6dd4fa26525d81b01d1474548bbdcec3908457a0559c5e114a04ebea990f"} Jul 18 12:35:20.885126 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.885079 2578 scope.go:117] "RemoveContainer" containerID="3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9" Jul 18 12:35:20.893748 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.893732 2578 scope.go:117] "RemoveContainer" containerID="3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9" Jul 18 12:35:20.894007 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:35:20.893986 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9\": container with ID starting with 3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9 not found: ID does not exist" containerID="3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9" Jul 18 12:35:20.894080 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.894018 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9"} err="failed to get container status \"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9\": rpc error: code = NotFound desc = could not find container \"3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9\": container with ID starting with 3b0c20aacfbcb9aa71781a506f08c67bf7485a9ccee45948bdc193f52d995cc9 not found: ID does not exist" Jul 18 12:35:20.906217 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.906197 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:35:20.911813 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.911791 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/sub-test-model-8b9f97-kserve-75557cfd77-j75nj"] Jul 18 12:35:20.948697 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.948675 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-dshm\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:20.948697 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.948694 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-home\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:20.948814 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.948702 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tls-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:20.948814 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.948710 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/481e5c09-af0e-49ca-9ca8-f9d12958bb69-tmp-dir\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:20.948814 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:20.948718 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkz4g\" (UniqueName: \"kubernetes.io/projected/481e5c09-af0e-49ca-9ca8-f9d12958bb69-kube-api-access-jkz4g\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:35:21.412038 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:21.412002 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" path="/var/lib/kubelet/pods/481e5c09-af0e-49ca-9ca8-f9d12958bb69/volumes" Jul 18 12:35:54.548739 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.548703 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:35:54.549166 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.549090 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" containerName="main" Jul 18 12:35:54.549166 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.549103 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" containerName="main" Jul 18 12:35:54.549252 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.549179 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="481e5c09-af0e-49ca-9ca8-f9d12958bb69" containerName="main" Jul 18 12:35:54.553292 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.553266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.555949 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.555861 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"kube-root-ca.crt\"" Jul 18 12:35:54.556828 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.556804 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"default-dockercfg-hq56s\"" Jul 18 12:35:54.556950 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.556805 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"openshift-service-ca.crt\"" Jul 18 12:35:54.556950 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.556855 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-8b9f97\"/\"rate-test-model-8b9f97-kserve-self-signed-certs\"" Jul 18 12:35:54.562637 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.562619 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:35:54.629704 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.629704 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.629921 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.629921 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.629921 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.629921 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.629848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsk48\" (UniqueName: \"kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730689 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730869 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730869 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730869 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730869 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.730869 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.730805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsk48\" (UniqueName: \"kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.731196 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.731174 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.731258 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.731237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.731310 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.731291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.733158 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.733126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.733379 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.733359 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.738452 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.738431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsk48\" (UniqueName: \"kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48\") pod \"rate-test-model-8b9f97-kserve-768d76dcc9-5697j\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.864561 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.864431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:35:54.996096 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:54.995925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:35:54.998843 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:35:54.998810 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod332b413f_177a_4311_ad63_0daa4a51f23f.slice/crio-3a025bfb033f7d54035fef85ed055c5e0de885997221c8cc0ef2878bd2462498 WatchSource:0}: Error finding container 3a025bfb033f7d54035fef85ed055c5e0de885997221c8cc0ef2878bd2462498: Status 404 returned error can't find the container with id 3a025bfb033f7d54035fef85ed055c5e0de885997221c8cc0ef2878bd2462498 Jul 18 12:35:55.060468 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:55.060443 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" event={"ID":"332b413f-177a-4311-ad63-0daa4a51f23f","Type":"ContainerStarted","Data":"3a025bfb033f7d54035fef85ed055c5e0de885997221c8cc0ef2878bd2462498"} Jul 18 12:35:56.068554 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:56.068518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" event={"ID":"332b413f-177a-4311-ad63-0daa4a51f23f","Type":"ContainerStarted","Data":"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d"} Jul 18 12:35:56.086134 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:35:56.086091 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" podStartSLOduration=2.086076595 podStartE2EDuration="2.086076595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 12:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:35:56.085041033 +0000 UTC m=+2825.157100536" watchObservedRunningTime="2026-07-18 12:35:56.086076595 +0000 UTC m=+2825.158136098" Jul 18 12:36:04.864586 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:04.864545 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:04.865008 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:04.864602 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:04.876875 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:04.876845 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:05.126661 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:05.126580 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:37.666943 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:37.666907 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:36:38.575283 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.575248 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108069047984"] Jul 18 12:36:38.578460 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.578443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:38.580697 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.580680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rf7pw\"" Jul 18 12:36:38.586494 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.586168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108069047984"] Jul 18 12:36:38.619870 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.619839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h59k\" (UniqueName: \"kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k\") pod \"test-curl-7717-140108069047984\" (UID: \"bdbe98e9-3692-4907-a71a-fc1918985bee\") " pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:38.720889 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.720852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h59k\" (UniqueName: \"kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k\") pod \"test-curl-7717-140108069047984\" (UID: \"bdbe98e9-3692-4907-a71a-fc1918985bee\") " pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:38.729828 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.729804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h59k\" (UniqueName: \"kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k\") pod \"test-curl-7717-140108069047984\" (UID: \"bdbe98e9-3692-4907-a71a-fc1918985bee\") " pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:38.889190 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:38.889098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:39.025915 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:39.025892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108069047984"] Jul 18 12:36:39.027882 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:39.027848 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdbe98e9_3692_4907_a71a_fc1918985bee.slice/crio-00383227cfb4dfcb496754fce50023683eb95fca3a06b0f54cb17429e9c720cc WatchSource:0}: Error finding container 00383227cfb4dfcb496754fce50023683eb95fca3a06b0f54cb17429e9c720cc: Status 404 returned error can't find the container with id 00383227cfb4dfcb496754fce50023683eb95fca3a06b0f54cb17429e9c720cc Jul 18 12:36:39.029131 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:39.029113 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 12:36:39.293358 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:39.293327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108069047984" event={"ID":"bdbe98e9-3692-4907-a71a-fc1918985bee","Type":"ContainerStarted","Data":"00383227cfb4dfcb496754fce50023683eb95fca3a06b0f54cb17429e9c720cc"} Jul 18 12:36:40.299258 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:40.299221 2578 generic.go:356] "Generic (PLEG): container finished" podID="bdbe98e9-3692-4907-a71a-fc1918985bee" containerID="40123c5a8dbc11f0437795a34498f337deab525bc801498b21d4dadb5776391e" exitCode=0 Jul 18 12:36:40.299697 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:40.299273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108069047984" event={"ID":"bdbe98e9-3692-4907-a71a-fc1918985bee","Type":"ContainerDied","Data":"40123c5a8dbc11f0437795a34498f337deab525bc801498b21d4dadb5776391e"} Jul 18 12:36:41.442790 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.442760 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:41.478878 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.478843 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108069047984_bdbe98e9-3692-4907-a71a-fc1918985bee/test-curl-7717-140108069047984/0.log" Jul 18 12:36:41.502752 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.502720 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108069047984"] Jul 18 12:36:41.506578 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.506554 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108069047984"] Jul 18 12:36:41.546868 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.546837 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h59k\" (UniqueName: \"kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k\") pod \"bdbe98e9-3692-4907-a71a-fc1918985bee\" (UID: \"bdbe98e9-3692-4907-a71a-fc1918985bee\") " Jul 18 12:36:41.549019 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.548996 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k" (OuterVolumeSpecName: "kube-api-access-8h59k") pod "bdbe98e9-3692-4907-a71a-fc1918985bee" (UID: "bdbe98e9-3692-4907-a71a-fc1918985bee"). InnerVolumeSpecName "kube-api-access-8h59k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:41.624631 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.624544 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108069061424"] Jul 18 12:36:41.624983 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.624965 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" containerName="test-curl-7717-140108069047984" Jul 18 12:36:41.624983 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.624983 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" containerName="test-curl-7717-140108069047984" Jul 18 12:36:41.625149 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.625062 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" containerName="test-curl-7717-140108069047984" Jul 18 12:36:41.628058 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.628025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:41.632894 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.632866 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108069061424"] Jul 18 12:36:41.648315 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.648287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cfs5\" (UniqueName: \"kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5\") pod \"test-curl-7717-140108069061424\" (UID: \"b62c1d53-889b-4810-af65-341bd6188e66\") " pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:41.648419 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.648406 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h59k\" (UniqueName: \"kubernetes.io/projected/bdbe98e9-3692-4907-a71a-fc1918985bee-kube-api-access-8h59k\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:41.748855 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.748822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cfs5\" (UniqueName: \"kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5\") pod \"test-curl-7717-140108069061424\" (UID: \"b62c1d53-889b-4810-af65-341bd6188e66\") " pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:41.757214 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.757188 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cfs5\" (UniqueName: \"kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5\") pod \"test-curl-7717-140108069061424\" (UID: \"b62c1d53-889b-4810-af65-341bd6188e66\") " pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:41.940084 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:41.940036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:42.067086 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:42.067021 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108069061424"] Jul 18 12:36:42.069201 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:42.069174 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb62c1d53_889b_4810_af65_341bd6188e66.slice/crio-5f406c4d654c686ee7f7dec1d457e7fadd1794ee771d38b54ece5875957e673e WatchSource:0}: Error finding container 5f406c4d654c686ee7f7dec1d457e7fadd1794ee771d38b54ece5875957e673e: Status 404 returned error can't find the container with id 5f406c4d654c686ee7f7dec1d457e7fadd1794ee771d38b54ece5875957e673e Jul 18 12:36:42.311095 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:42.310976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108069061424" event={"ID":"b62c1d53-889b-4810-af65-341bd6188e66","Type":"ContainerStarted","Data":"5f406c4d654c686ee7f7dec1d457e7fadd1794ee771d38b54ece5875957e673e"} Jul 18 12:36:42.312256 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:42.312237 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:42.312391 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:42.312277 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00383227cfb4dfcb496754fce50023683eb95fca3a06b0f54cb17429e9c720cc" Jul 18 12:36:42.322844 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:42.322817 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7717-140108069047984\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:43.319496 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:43.319457 2578 generic.go:356] "Generic (PLEG): container finished" podID="b62c1d53-889b-4810-af65-341bd6188e66" containerID="327c48cf121e54d017e92d66b83248ceed1eb08e83c8a91bde3a4a1905a86e8d" exitCode=0 Jul 18 12:36:43.319893 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:43.319512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108069061424" event={"ID":"b62c1d53-889b-4810-af65-341bd6188e66","Type":"ContainerDied","Data":"327c48cf121e54d017e92d66b83248ceed1eb08e83c8a91bde3a4a1905a86e8d"} Jul 18 12:36:43.334058 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:43.334001 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7717-140108069047984\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" pod="opendatahub/test-curl-7717-140108069047984" Jul 18 12:36:43.411662 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:43.411629 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbe98e9-3692-4907-a71a-fc1918985bee" path="/var/lib/kubelet/pods/bdbe98e9-3692-4907-a71a-fc1918985bee/volumes" Jul 18 12:36:44.455524 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.455501 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:44.473573 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.473543 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108069061424_b62c1d53-889b-4810-af65-341bd6188e66/test-curl-7717-140108069061424/0.log" Jul 18 12:36:44.495174 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.495147 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108069061424"] Jul 18 12:36:44.497233 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.497208 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108069061424"] Jul 18 12:36:44.572685 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.572657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cfs5\" (UniqueName: \"kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5\") pod \"b62c1d53-889b-4810-af65-341bd6188e66\" (UID: \"b62c1d53-889b-4810-af65-341bd6188e66\") " Jul 18 12:36:44.574744 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.574719 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5" (OuterVolumeSpecName: "kube-api-access-2cfs5") pod "b62c1d53-889b-4810-af65-341bd6188e66" (UID: "b62c1d53-889b-4810-af65-341bd6188e66"). InnerVolumeSpecName "kube-api-access-2cfs5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:44.657496 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.657462 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:44.657866 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.657848 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b62c1d53-889b-4810-af65-341bd6188e66" containerName="test-curl-7717-140108069061424" Jul 18 12:36:44.657945 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.657868 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b62c1d53-889b-4810-af65-341bd6188e66" containerName="test-curl-7717-140108069061424" Jul 18 12:36:44.658007 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.657993 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b62c1d53-889b-4810-af65-341bd6188e66" containerName="test-curl-7717-140108069061424" Jul 18 12:36:44.660986 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.660967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:44.667250 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.667225 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:44.673332 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.673308 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cfs5\" (UniqueName: \"kubernetes.io/projected/b62c1d53-889b-4810-af65-341bd6188e66-kube-api-access-2cfs5\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:44.774314 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.774278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx5bg\" (UniqueName: \"kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg\") pod \"test-curl-7717-140108068975504\" (UID: \"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:44.875351 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.875260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx5bg\" (UniqueName: \"kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg\") pod \"test-curl-7717-140108068975504\" (UID: \"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:44.883674 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.883645 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx5bg\" (UniqueName: \"kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg\") pod \"test-curl-7717-140108068975504\" (UID: \"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:44.970933 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:44.970899 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:45.101456 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:45.101430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:45.103309 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:45.103268 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f5a1bcd_2e11_4e71_a1ac_1b8f6318828a.slice/crio-20b7bb7dc216bac3cb9032102f788af73b214cf376b6ffa4e76a9ddea206ef9f WatchSource:0}: Error finding container 20b7bb7dc216bac3cb9032102f788af73b214cf376b6ffa4e76a9ddea206ef9f: Status 404 returned error can't find the container with id 20b7bb7dc216bac3cb9032102f788af73b214cf376b6ffa4e76a9ddea206ef9f Jul 18 12:36:45.331867 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:45.331828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068975504" event={"ID":"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a","Type":"ContainerStarted","Data":"20b7bb7dc216bac3cb9032102f788af73b214cf376b6ffa4e76a9ddea206ef9f"} Jul 18 12:36:45.333429 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:45.333401 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:45.333551 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:45.333428 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f406c4d654c686ee7f7dec1d457e7fadd1794ee771d38b54ece5875957e673e" Jul 18 12:36:45.343369 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:45.343338 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7717-140108069061424\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="b62c1d53-889b-4810-af65-341bd6188e66" pod="opendatahub/test-curl-7717-140108069061424" Jul 18 12:36:45.412144 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:45.412111 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b62c1d53-889b-4810-af65-341bd6188e66" path="/var/lib/kubelet/pods/b62c1d53-889b-4810-af65-341bd6188e66/volumes" Jul 18 12:36:46.340187 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:46.340149 2578 generic.go:356] "Generic (PLEG): container finished" podID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" containerID="d9029c5c0579136a435f31f8e78f4279cf3edc47eeb3624ab55453ca58d96341" exitCode=0 Jul 18 12:36:46.340574 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:46.340194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068975504" event={"ID":"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a","Type":"ContainerDied","Data":"d9029c5c0579136a435f31f8e78f4279cf3edc47eeb3624ab55453ca58d96341"} Jul 18 12:36:47.487413 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.487386 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:47.506026 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.506003 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108068975504_6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a/test-curl-7717-140108068975504/0.log" Jul 18 12:36:47.528494 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.528461 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:47.531733 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.531707 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:47.597881 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.597787 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx5bg\" (UniqueName: \"kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg\") pod \"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a\" (UID: \"6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a\") " Jul 18 12:36:47.599906 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.599879 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg" (OuterVolumeSpecName: "kube-api-access-lx5bg") pod "6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" (UID: "6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a"). InnerVolumeSpecName "kube-api-access-lx5bg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:47.651054 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.651024 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:47.651388 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.651374 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" containerName="test-curl-7717-140108068975504" Jul 18 12:36:47.651431 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.651389 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" containerName="test-curl-7717-140108068975504" Jul 18 12:36:47.651471 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.651449 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" containerName="test-curl-7717-140108068975504" Jul 18 12:36:47.654352 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.654336 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:47.661198 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.661174 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:47.699062 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.699013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds2xk\" (UniqueName: \"kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk\") pod \"test-curl-7717-140108068975504\" (UID: \"745aff77-1a00-4da5-8710-d037e1d6d5c5\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:47.699228 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.699185 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx5bg\" (UniqueName: \"kubernetes.io/projected/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a-kube-api-access-lx5bg\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:47.800385 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.800348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds2xk\" (UniqueName: \"kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk\") pod \"test-curl-7717-140108068975504\" (UID: \"745aff77-1a00-4da5-8710-d037e1d6d5c5\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:47.808291 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.808268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds2xk\" (UniqueName: \"kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk\") pod \"test-curl-7717-140108068975504\" (UID: \"745aff77-1a00-4da5-8710-d037e1d6d5c5\") " pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:47.964591 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:47.964564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:48.094430 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:48.094405 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:48.095882 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:48.095858 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod745aff77_1a00_4da5_8710_d037e1d6d5c5.slice/crio-8a0bfd322e0d691c40a95796f736cc14bd28ef14fb564618625fe4e06d1dba55 WatchSource:0}: Error finding container 8a0bfd322e0d691c40a95796f736cc14bd28ef14fb564618625fe4e06d1dba55: Status 404 returned error can't find the container with id 8a0bfd322e0d691c40a95796f736cc14bd28ef14fb564618625fe4e06d1dba55 Jul 18 12:36:48.351895 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:48.351822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068975504" event={"ID":"745aff77-1a00-4da5-8710-d037e1d6d5c5","Type":"ContainerStarted","Data":"8a0bfd322e0d691c40a95796f736cc14bd28ef14fb564618625fe4e06d1dba55"} Jul 18 12:36:48.353006 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:48.352985 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20b7bb7dc216bac3cb9032102f788af73b214cf376b6ffa4e76a9ddea206ef9f" Jul 18 12:36:48.353155 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:48.353024 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:48.364819 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:48.364584 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7717-140108068975504" oldPodUID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" Jul 18 12:36:49.359513 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:49.359476 2578 generic.go:356] "Generic (PLEG): container finished" podID="745aff77-1a00-4da5-8710-d037e1d6d5c5" containerID="a7cfe1fe84bab74fd19567c89555694bfc0fed435535a59658b74b4fb9618226" exitCode=0 Jul 18 12:36:49.359894 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:49.359562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068975504" event={"ID":"745aff77-1a00-4da5-8710-d037e1d6d5c5","Type":"ContainerDied","Data":"a7cfe1fe84bab74fd19567c89555694bfc0fed435535a59658b74b4fb9618226"} Jul 18 12:36:49.413380 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:49.413348 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a" path="/var/lib/kubelet/pods/6f5a1bcd-2e11-4e71-a1ac-1b8f6318828a/volumes" Jul 18 12:36:50.494952 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.494929 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:50.515303 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.515273 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108068975504_745aff77-1a00-4da5-8710-d037e1d6d5c5/test-curl-7717-140108068975504/0.log" Jul 18 12:36:50.537318 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.537283 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:50.540872 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.540844 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108068975504"] Jul 18 12:36:50.624259 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.624169 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds2xk\" (UniqueName: \"kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk\") pod \"745aff77-1a00-4da5-8710-d037e1d6d5c5\" (UID: \"745aff77-1a00-4da5-8710-d037e1d6d5c5\") " Jul 18 12:36:50.626217 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.626182 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk" (OuterVolumeSpecName: "kube-api-access-ds2xk") pod "745aff77-1a00-4da5-8710-d037e1d6d5c5" (UID: "745aff77-1a00-4da5-8710-d037e1d6d5c5"). InnerVolumeSpecName "kube-api-access-ds2xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:50.656081 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.656039 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:50.656428 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.656415 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" containerName="test-curl-7717-140108068975504" Jul 18 12:36:50.656479 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.656429 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" containerName="test-curl-7717-140108068975504" Jul 18 12:36:50.656519 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.656486 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" containerName="test-curl-7717-140108068975504" Jul 18 12:36:50.659471 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.659456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:50.663191 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.663167 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:50.724981 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.724956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6vp2\" (UniqueName: \"kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2\") pod \"test-curl-7717-140108068976512\" (UID: \"e7f28062-7049-4731-b31e-74b7c109441d\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:50.725120 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.724994 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds2xk\" (UniqueName: \"kubernetes.io/projected/745aff77-1a00-4da5-8710-d037e1d6d5c5-kube-api-access-ds2xk\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:50.826242 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.826212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6vp2\" (UniqueName: \"kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2\") pod \"test-curl-7717-140108068976512\" (UID: \"e7f28062-7049-4731-b31e-74b7c109441d\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:50.834359 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.834330 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6vp2\" (UniqueName: \"kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2\") pod \"test-curl-7717-140108068976512\" (UID: \"e7f28062-7049-4731-b31e-74b7c109441d\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:50.970186 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:50.970151 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:51.342277 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.342254 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:51.343998 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:51.343973 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7f28062_7049_4731_b31e_74b7c109441d.slice/crio-535e034baab5d68669595b6d1dcbab22dee3f47a7ff3993916dd619eb0845364 WatchSource:0}: Error finding container 535e034baab5d68669595b6d1dcbab22dee3f47a7ff3993916dd619eb0845364: Status 404 returned error can't find the container with id 535e034baab5d68669595b6d1dcbab22dee3f47a7ff3993916dd619eb0845364 Jul 18 12:36:51.372062 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.372029 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:51.372188 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.372035 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a0bfd322e0d691c40a95796f736cc14bd28ef14fb564618625fe4e06d1dba55" Jul 18 12:36:51.373266 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.373244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068976512" event={"ID":"e7f28062-7049-4731-b31e-74b7c109441d","Type":"ContainerStarted","Data":"535e034baab5d68669595b6d1dcbab22dee3f47a7ff3993916dd619eb0845364"} Jul 18 12:36:51.385651 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:51.385623 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7717-140108068975504\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:51.410381 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:51.410355 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7717-140108068975504\" is forbidden: User \"system:node:ip-10-0-130-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-242.ec2.internal' and this object" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" pod="opendatahub/test-curl-7717-140108068975504" Jul 18 12:36:51.412018 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.411997 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="745aff77-1a00-4da5-8710-d037e1d6d5c5" path="/var/lib/kubelet/pods/745aff77-1a00-4da5-8710-d037e1d6d5c5/volumes" Jul 18 12:36:51.744648 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:51.744624 2578 scope.go:117] "RemoveContainer" containerID="10d6dc6cd5fc6e09bfe0cc714f9223bf04cac2303171e0435c2d7a69857f99db" Jul 18 12:36:52.380129 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:52.380099 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7f28062-7049-4731-b31e-74b7c109441d" containerID="8974d2ebdac1bdb935b4f27656f0ef37e248d7625db4d1f5e2f9b1d1d7b96ee7" exitCode=0 Jul 18 12:36:52.380305 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:52.380189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068976512" event={"ID":"e7f28062-7049-4731-b31e-74b7c109441d","Type":"ContainerDied","Data":"8974d2ebdac1bdb935b4f27656f0ef37e248d7625db4d1f5e2f9b1d1d7b96ee7"} Jul 18 12:36:52.687619 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:52.687575 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" podUID="332b413f-177a-4311-ad63-0daa4a51f23f" containerName="main" containerID="cri-o://b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d" gracePeriod=45 Jul 18 12:36:52.934098 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:52.934073 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:53.048288 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048203 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048288 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048257 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsk48\" (UniqueName: \"kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048293 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048309 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048336 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048383 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir\") pod \"332b413f-177a-4311-ad63-0daa4a51f23f\" (UID: \"332b413f-177a-4311-ad63-0daa4a51f23f\") " Jul 18 12:36:53.048528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048515 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache" (OuterVolumeSpecName: "model-cache") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:36:53.048792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048569 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home" (OuterVolumeSpecName: "home") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:36:53.048792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048673 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-model-cache\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.048792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048675 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:36:53.048792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.048693 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-home\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.050498 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.050467 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm" (OuterVolumeSpecName: "dshm") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 12:36:53.050632 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.050508 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48" (OuterVolumeSpecName: "kube-api-access-vsk48") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "kube-api-access-vsk48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:53.050632 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.050585 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "332b413f-177a-4311-ad63-0daa4a51f23f" (UID: "332b413f-177a-4311-ad63-0daa4a51f23f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 12:36:53.150129 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.150086 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsk48\" (UniqueName: \"kubernetes.io/projected/332b413f-177a-4311-ad63-0daa4a51f23f-kube-api-access-vsk48\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.150129 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.150123 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-dshm\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.150129 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.150133 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/332b413f-177a-4311-ad63-0daa4a51f23f-tls-certs\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.150357 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.150164 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/332b413f-177a-4311-ad63-0daa4a51f23f-tmp-dir\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.386784 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.386693 2578 generic.go:356] "Generic (PLEG): container finished" podID="332b413f-177a-4311-ad63-0daa4a51f23f" containerID="b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d" exitCode=0 Jul 18 12:36:53.386784 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.386743 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" Jul 18 12:36:53.387014 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.386787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" event={"ID":"332b413f-177a-4311-ad63-0daa4a51f23f","Type":"ContainerDied","Data":"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d"} Jul 18 12:36:53.387014 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.386835 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j" event={"ID":"332b413f-177a-4311-ad63-0daa4a51f23f","Type":"ContainerDied","Data":"3a025bfb033f7d54035fef85ed055c5e0de885997221c8cc0ef2878bd2462498"} Jul 18 12:36:53.387014 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.386857 2578 scope.go:117] "RemoveContainer" containerID="b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d" Jul 18 12:36:53.396274 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.396249 2578 scope.go:117] "RemoveContainer" containerID="b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d" Jul 18 12:36:53.396633 ip-10-0-130-242 kubenswrapper[2578]: E0718 12:36:53.396575 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d\": container with ID starting with b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d not found: ID does not exist" containerID="b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d" Jul 18 12:36:53.396633 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.396610 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d"} err="failed to get container status \"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d\": rpc error: code = NotFound desc = could not find container \"b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d\": container with ID starting with b0041d0a732dbb21d5173f497ee64b3e2f31b5fed899644bf10b2e1eafa0758d not found: ID does not exist" Jul 18 12:36:53.414170 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.414118 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:36:53.415370 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.415342 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-8b9f97/rate-test-model-8b9f97-kserve-768d76dcc9-5697j"] Jul 18 12:36:53.510640 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.510615 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:53.529382 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.529354 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108068976512_e7f28062-7049-4731-b31e-74b7c109441d/test-curl-7717-140108068976512/0.log" Jul 18 12:36:53.552844 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.552810 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:53.553011 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.552993 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6vp2\" (UniqueName: \"kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2\") pod \"e7f28062-7049-4731-b31e-74b7c109441d\" (UID: \"e7f28062-7049-4731-b31e-74b7c109441d\") " Jul 18 12:36:53.555019 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.555000 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2" (OuterVolumeSpecName: "kube-api-access-w6vp2") pod "e7f28062-7049-4731-b31e-74b7c109441d" (UID: "e7f28062-7049-4731-b31e-74b7c109441d"). InnerVolumeSpecName "kube-api-access-w6vp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:53.558099 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.558076 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:53.654564 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.654485 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6vp2\" (UniqueName: \"kubernetes.io/projected/e7f28062-7049-4731-b31e-74b7c109441d-kube-api-access-w6vp2\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:53.672224 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672192 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:53.672529 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672516 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f28062-7049-4731-b31e-74b7c109441d" containerName="test-curl-7717-140108068976512" Jul 18 12:36:53.672576 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672531 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f28062-7049-4731-b31e-74b7c109441d" containerName="test-curl-7717-140108068976512" Jul 18 12:36:53.672576 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672541 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332b413f-177a-4311-ad63-0daa4a51f23f" containerName="main" Jul 18 12:36:53.672576 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672547 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="332b413f-177a-4311-ad63-0daa4a51f23f" containerName="main" Jul 18 12:36:53.672666 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672616 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f28062-7049-4731-b31e-74b7c109441d" containerName="test-curl-7717-140108068976512" Jul 18 12:36:53.672666 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.672624 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="332b413f-177a-4311-ad63-0daa4a51f23f" containerName="main" Jul 18 12:36:53.675528 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.675513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:53.679903 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.679877 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:53.755167 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.755127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rm4\" (UniqueName: \"kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4\") pod \"test-curl-7717-140108068976512\" (UID: \"79b242cc-f60f-4a2a-98ab-da7fcca4fd39\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:53.855628 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.855591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rm4\" (UniqueName: \"kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4\") pod \"test-curl-7717-140108068976512\" (UID: \"79b242cc-f60f-4a2a-98ab-da7fcca4fd39\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:53.863937 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.863913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rm4\" (UniqueName: \"kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4\") pod \"test-curl-7717-140108068976512\" (UID: \"79b242cc-f60f-4a2a-98ab-da7fcca4fd39\") " pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:53.985830 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:53.985803 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:54.322956 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:54.322931 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:54.324428 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:54.324400 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79b242cc_f60f_4a2a_98ab_da7fcca4fd39.slice/crio-ad29592ee538a0fa0bba70b54818ceb5dabd34141f15f20ea282a41f15ddb7c0 WatchSource:0}: Error finding container ad29592ee538a0fa0bba70b54818ceb5dabd34141f15f20ea282a41f15ddb7c0: Status 404 returned error can't find the container with id ad29592ee538a0fa0bba70b54818ceb5dabd34141f15f20ea282a41f15ddb7c0 Jul 18 12:36:54.394164 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:54.394140 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="535e034baab5d68669595b6d1dcbab22dee3f47a7ff3993916dd619eb0845364" Jul 18 12:36:54.394164 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:54.394143 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:54.395303 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:54.395280 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068976512" event={"ID":"79b242cc-f60f-4a2a-98ab-da7fcca4fd39","Type":"ContainerStarted","Data":"ad29592ee538a0fa0bba70b54818ceb5dabd34141f15f20ea282a41f15ddb7c0"} Jul 18 12:36:54.405378 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:54.405354 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7717-140108068976512" oldPodUID="e7f28062-7049-4731-b31e-74b7c109441d" podUID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" Jul 18 12:36:55.403213 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:55.403180 2578 generic.go:356] "Generic (PLEG): container finished" podID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" containerID="491a19b289e168fc9ba7ffee7df34a9698e58bcb11801e37511ed38a65e8f395" exitCode=0 Jul 18 12:36:55.403587 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:55.403268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140108068976512" event={"ID":"79b242cc-f60f-4a2a-98ab-da7fcca4fd39","Type":"ContainerDied","Data":"491a19b289e168fc9ba7ffee7df34a9698e58bcb11801e37511ed38a65e8f395"} Jul 18 12:36:55.411830 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:55.411807 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332b413f-177a-4311-ad63-0daa4a51f23f" path="/var/lib/kubelet/pods/332b413f-177a-4311-ad63-0daa4a51f23f/volumes" Jul 18 12:36:55.412163 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:55.412150 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f28062-7049-4731-b31e-74b7c109441d" path="/var/lib/kubelet/pods/e7f28062-7049-4731-b31e-74b7c109441d/volumes" Jul 18 12:36:56.543994 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.543968 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:56.562617 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.562589 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140108068976512_79b242cc-f60f-4a2a-98ab-da7fcca4fd39/test-curl-7717-140108068976512/0.log" Jul 18 12:36:56.584533 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.584503 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:56.588563 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.588538 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140108068976512"] Jul 18 12:36:56.680572 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.680547 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6rm4\" (UniqueName: \"kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4\") pod \"79b242cc-f60f-4a2a-98ab-da7fcca4fd39\" (UID: \"79b242cc-f60f-4a2a-98ab-da7fcca4fd39\") " Jul 18 12:36:56.682687 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.682660 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4" (OuterVolumeSpecName: "kube-api-access-w6rm4") pod "79b242cc-f60f-4a2a-98ab-da7fcca4fd39" (UID: "79b242cc-f60f-4a2a-98ab-da7fcca4fd39"). InnerVolumeSpecName "kube-api-access-w6rm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 12:36:56.768719 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.768679 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz"] Jul 18 12:36:56.769111 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.769093 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" containerName="test-curl-7717-140108068976512" Jul 18 12:36:56.769207 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.769112 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" containerName="test-curl-7717-140108068976512" Jul 18 12:36:56.769265 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.769214 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" containerName="test-curl-7717-140108068976512" Jul 18 12:36:56.772248 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.772228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:56.774313 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.774292 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 12:36:56.774413 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.774326 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cmjp2\"" Jul 18 12:36:56.778849 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.778794 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz"] Jul 18 12:36:56.781531 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.781509 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6rm4\" (UniqueName: \"kubernetes.io/projected/79b242cc-f60f-4a2a-98ab-da7fcca4fd39-kube-api-access-w6rm4\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:36:56.882024 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.881994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz\" (UID: \"6b70a532-9a2c-473e-a970-dc0b3b7102b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:56.982513 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.982439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz\" (UID: \"6b70a532-9a2c-473e-a970-dc0b3b7102b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:56.983008 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:56.982988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz\" (UID: \"6b70a532-9a2c-473e-a970-dc0b3b7102b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:57.084922 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.084891 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:57.216353 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.216328 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz"] Jul 18 12:36:57.218169 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:36:57.218144 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b70a532_9a2c_473e_a970_dc0b3b7102b1.slice/crio-7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292 WatchSource:0}: Error finding container 7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292: Status 404 returned error can't find the container with id 7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292 Jul 18 12:36:57.414134 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.414101 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79b242cc-f60f-4a2a-98ab-da7fcca4fd39" path="/var/lib/kubelet/pods/79b242cc-f60f-4a2a-98ab-da7fcca4fd39/volumes" Jul 18 12:36:57.418390 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.418367 2578 scope.go:117] "RemoveContainer" containerID="491a19b289e168fc9ba7ffee7df34a9698e58bcb11801e37511ed38a65e8f395" Jul 18 12:36:57.418505 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.418371 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140108068976512" Jul 18 12:36:57.419723 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:57.419699 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" event={"ID":"6b70a532-9a2c-473e-a970-dc0b3b7102b1","Type":"ContainerStarted","Data":"7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292"} Jul 18 12:36:58.427514 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:58.427480 2578 generic.go:356] "Generic (PLEG): container finished" podID="6b70a532-9a2c-473e-a970-dc0b3b7102b1" containerID="0f3299fbec0ba6847c448fc26b0dbe27893e3df17599e2e4b03948b0f99cda6f" exitCode=0 Jul 18 12:36:58.428000 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:58.427571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" event={"ID":"6b70a532-9a2c-473e-a970-dc0b3b7102b1","Type":"ContainerDied","Data":"0f3299fbec0ba6847c448fc26b0dbe27893e3df17599e2e4b03948b0f99cda6f"} Jul 18 12:36:59.563973 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:59.563945 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:36:59.704075 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:59.703966 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca\") pod \"6b70a532-9a2c-473e-a970-dc0b3b7102b1\" (UID: \"6b70a532-9a2c-473e-a970-dc0b3b7102b1\") " Jul 18 12:36:59.704367 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:59.704347 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b70a532-9a2c-473e-a970-dc0b3b7102b1" (UID: "6b70a532-9a2c-473e-a970-dc0b3b7102b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 12:36:59.805529 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:36:59.805494 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b70a532-9a2c-473e-a970-dc0b3b7102b1-maas-api-service-ca\") on node \"ip-10-0-130-242.ec2.internal\" DevicePath \"\"" Jul 18 12:37:00.439332 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:00.439304 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" Jul 18 12:37:00.439516 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:00.439348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz" event={"ID":"6b70a532-9a2c-473e-a970-dc0b3b7102b1","Type":"ContainerDied","Data":"7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292"} Jul 18 12:37:00.439516 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:00.439396 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b76c3887933c0346fcb0fa7b8b411f4774e9bfd22fe9a744f8fe463dc34e292" Jul 18 12:37:11.859383 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:11.859346 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz"] Jul 18 12:37:11.861587 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:11.861564 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7a27b1-tqrpz"] Jul 18 12:37:13.411650 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:13.411618 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b70a532-9a2c-473e-a970-dc0b3b7102b1" path="/var/lib/kubelet/pods/6b70a532-9a2c-473e-a970-dc0b3b7102b1/volumes" Jul 18 12:37:41.359880 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:41.359845 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6844644696-942js_235fa772-bfa8-47f4-be66-eb551e598cd7/authorino/0.log" Jul 18 12:37:47.359313 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:47.359284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbb59dc99-rwbbk_bc49d736-1e38-4146-bf63-53333de54985/copy-manifests/0.log" Jul 18 12:37:47.371475 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:47.371445 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbb59dc99-rwbbk_bc49d736-1e38-4146-bf63-53333de54985/manager/0.log" Jul 18 12:37:48.044792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:48.044762 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739630-t2k6h_bf8c51fe-badf-40b9-8cc1-0167ecc9747d/cleanup/0.log" Jul 18 12:37:48.915245 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:48.915218 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/pull/0.log" Jul 18 12:37:48.921624 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:48.921602 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/extract/0.log" Jul 18 12:37:48.927445 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:48.927427 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/util/0.log" Jul 18 12:37:49.037178 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.037152 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/util/0.log" Jul 18 12:37:49.044107 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.044072 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/pull/0.log" Jul 18 12:37:49.050496 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.050479 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/extract/0.log" Jul 18 12:37:49.164247 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.164215 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/util/0.log" Jul 18 12:37:49.170861 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.170841 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/pull/0.log" Jul 18 12:37:49.177067 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.177020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/extract/0.log" Jul 18 12:37:49.285145 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.285118 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/util/0.log" Jul 18 12:37:49.290783 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.290763 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/pull/0.log" Jul 18 12:37:49.296526 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.296485 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/extract/0.log" Jul 18 12:37:49.413832 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.413791 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6844644696-942js_235fa772-bfa8-47f4-be66-eb551e598cd7/authorino/0.log" Jul 18 12:37:49.738198 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.738172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tw8l4_8f001dbb-89c2-49c0-bfc2-4ba18a55cf24/kuadrant-console-plugin/0.log" Jul 18 12:37:49.963820 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:49.963791 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-whl89_3d46cbd9-8e7a-43da-a619-94ab8df8ad3e/manager/0.log" Jul 18 12:37:50.765539 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:50.765505 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677848b8b7-b9vvk_dfa2a5d1-8057-4640-ae65-648fd313b181/kube-auth-proxy/0.log" Jul 18 12:37:51.212476 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:51.212447 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4kgb4_9365b5e2-a70e-42c0-8a7b-76b694c05504/payload-processing/0.log" Jul 18 12:37:51.858642 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:37:51.858617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-c44q7_17ec13aa-16d3-46c8-881e-ee76829abe49/main/0.log" Jul 18 12:38:00.643072 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:00.643024 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ndsm5_c73af55a-7e9a-416b-8144-38477c907ba0/global-pull-secret-syncer/0.log" Jul 18 12:38:00.805966 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:00.805937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7c5sk_a0b12eaf-f8e1-4431-9f83-c6ecb863b32c/konnectivity-agent/0.log" Jul 18 12:38:00.855954 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:00.855929 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-242.ec2.internal_5dcb4fafed7c79e7e7353434bd1881be/haproxy/0.log" Jul 18 12:38:05.656909 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.656884 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/extract/0.log" Jul 18 12:38:05.676497 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.676457 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/util/0.log" Jul 18 12:38:05.694462 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.694441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hr6g7_199bca4c-fab0-4473-8a56-77bf78e51f2a/pull/0.log" Jul 18 12:38:05.733386 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.733351 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/extract/0.log" Jul 18 12:38:05.751805 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.751779 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/util/0.log" Jul 18 12:38:05.769707 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.769686 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0nwtv6_3d822b8d-0156-4f21-b5c7-714f83053334/pull/0.log" Jul 18 12:38:05.792297 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.792280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/extract/0.log" Jul 18 12:38:05.809856 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.809839 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/util/0.log" Jul 18 12:38:05.828382 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.828358 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734vlmw_9b9ae344-5a95-48f9-b0a9-a9a650ce9bad/pull/0.log" Jul 18 12:38:05.850876 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.850856 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/extract/0.log" Jul 18 12:38:05.873175 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.873149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/util/0.log" Jul 18 12:38:05.892232 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:05.892212 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s6p8h_2f27f110-5bc4-4f3e-bf0b-8b2662d15e82/pull/0.log" Jul 18 12:38:06.219785 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:06.219754 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6844644696-942js_235fa772-bfa8-47f4-be66-eb551e598cd7/authorino/0.log" Jul 18 12:38:06.305385 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:06.305354 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tw8l4_8f001dbb-89c2-49c0-bfc2-4ba18a55cf24/kuadrant-console-plugin/0.log" Jul 18 12:38:06.489833 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:06.489734 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-whl89_3d46cbd9-8e7a-43da-a619-94ab8df8ad3e/manager/0.log" Jul 18 12:38:08.315658 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.315613 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-2fmbx_3c5caa8a-0f0d-4b5a-a096-2a7c12d5a4a9/cluster-monitoring-operator/0.log" Jul 18 12:38:08.442423 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.442397 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-w68zk_58a2b2fa-c547-49e8-b2a4-dc82281374a8/monitoring-plugin/0.log" Jul 18 12:38:08.468175 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.468149 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k5m2_c6a38575-245e-4416-9f8d-b904b2fa815e/node-exporter/0.log" Jul 18 12:38:08.486982 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.486948 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k5m2_c6a38575-245e-4416-9f8d-b904b2fa815e/kube-rbac-proxy/0.log" Jul 18 12:38:08.509941 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.509918 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2k5m2_c6a38575-245e-4416-9f8d-b904b2fa815e/init-textfile/0.log" Jul 18 12:38:08.934218 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.934180 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-zbvsk_c290effa-8784-4b77-8755-482d2cde809c/prometheus-operator/0.log" Jul 18 12:38:08.950205 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.950169 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-zbvsk_c290effa-8784-4b77-8755-482d2cde809c/kube-rbac-proxy/0.log" Jul 18 12:38:08.978220 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:08.978194 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-dlmbz_adc5ba80-f709-4803-8e30-c6a1828ef6eb/prometheus-operator-admission-webhook/0.log" Jul 18 12:38:09.101629 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.101597 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/thanos-query/0.log" Jul 18 12:38:09.120589 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.120565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/kube-rbac-proxy-web/0.log" Jul 18 12:38:09.140471 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.140434 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/kube-rbac-proxy/0.log" Jul 18 12:38:09.159392 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.159375 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/prom-label-proxy/0.log" Jul 18 12:38:09.179878 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.179845 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/kube-rbac-proxy-rules/0.log" Jul 18 12:38:09.198301 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:09.198235 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59bf87698d-rdrvx_12707f1a-abc1-4108-8ff2-45ee0879c600/kube-rbac-proxy-metrics/0.log" Jul 18 12:38:10.410544 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:10.410516 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-dc5cb_5d2213a6-b613-4961-9d5d-6b483a05b011/networking-console-plugin/0.log" Jul 18 12:38:11.003096 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.003057 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk"] Jul 18 12:38:11.003419 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.003406 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b70a532-9a2c-473e-a970-dc0b3b7102b1" containerName="revoke-keys" Jul 18 12:38:11.003463 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.003420 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b70a532-9a2c-473e-a970-dc0b3b7102b1" containerName="revoke-keys" Jul 18 12:38:11.003504 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.003495 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b70a532-9a2c-473e-a970-dc0b3b7102b1" containerName="revoke-keys" Jul 18 12:38:11.006965 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.006945 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.009323 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.009306 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6lsxg\"/\"kube-root-ca.crt\"" Jul 18 12:38:11.010105 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.010091 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6lsxg\"/\"default-dockercfg-fk6km\"" Jul 18 12:38:11.010105 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.010100 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6lsxg\"/\"openshift-service-ca.crt\"" Jul 18 12:38:11.016074 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.016040 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk"] Jul 18 12:38:11.088625 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.088591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52zk5\" (UniqueName: \"kubernetes.io/projected/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-kube-api-access-52zk5\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.088792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.088652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-lib-modules\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.088792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.088717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-podres\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.088792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.088784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-sys\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.088901 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.088806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-proc\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189698 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-lib-modules\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189865 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-podres\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189865 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-sys\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189865 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-proc\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189865 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52zk5\" (UniqueName: \"kubernetes.io/projected/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-kube-api-access-52zk5\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.189865 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-podres\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.190120 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-lib-modules\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.190120 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189902 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-sys\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.190120 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.189910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-proc\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.198007 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.197988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52zk5\" (UniqueName: \"kubernetes.io/projected/89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8-kube-api-access-52zk5\") pod \"perf-node-gather-daemonset-5q8zk\" (UID: \"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8\") " pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.318754 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.318665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.450987 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.450961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk"] Jul 18 12:38:11.452826 ip-10-0-130-242 kubenswrapper[2578]: W0718 12:38:11.452799 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod89fc1b3b_f674_4a7b_b7a6_dc79f3f586d8.slice/crio-6b568fe29fe4de5b479ce8fc71e06cafa48fcc2f3d79b73a57f19162b65813f2 WatchSource:0}: Error finding container 6b568fe29fe4de5b479ce8fc71e06cafa48fcc2f3d79b73a57f19162b65813f2: Status 404 returned error can't find the container with id 6b568fe29fe4de5b479ce8fc71e06cafa48fcc2f3d79b73a57f19162b65813f2 Jul 18 12:38:11.679058 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.679026 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bf56d4ccb-s7jtc_24686a53-3069-4793-82b7-ca71a56cecf4/console/0.log" Jul 18 12:38:11.824556 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.824523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" event={"ID":"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8","Type":"ContainerStarted","Data":"b25c6069b02e7d3343f92a6549dbe3c9a9e17ad444b9719183dce2dec69c7af9"} Jul 18 12:38:11.824754 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.824571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" event={"ID":"89fc1b3b-f674-4a7b-b7a6-dc79f3f586d8","Type":"ContainerStarted","Data":"6b568fe29fe4de5b479ce8fc71e06cafa48fcc2f3d79b73a57f19162b65813f2"} Jul 18 12:38:11.824754 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.824616 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:11.840398 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:11.840360 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" podStartSLOduration=1.8403481510000002 podStartE2EDuration="1.840348151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 12:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 12:38:11.839455582 +0000 UTC m=+2960.911515087" watchObservedRunningTime="2026-07-18 12:38:11.840348151 +0000 UTC m=+2960.912407653" Jul 18 12:38:13.332304 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:13.332254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mm94p_e156db75-f982-4a5e-8cbe-8b06b24aa23f/dns/0.log" Jul 18 12:38:13.351249 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:13.351218 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mm94p_e156db75-f982-4a5e-8cbe-8b06b24aa23f/kube-rbac-proxy/0.log" Jul 18 12:38:13.479947 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:13.479918 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jt2z9_1e7deda4-422f-4caa-93f0-876b65fdb720/dns-node-resolver/0.log" Jul 18 12:38:14.046208 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:14.046122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-49tjp_65fca35e-cf89-44f8-ac27-6d26f9b1753a/node-ca/0.log" Jul 18 12:38:15.266345 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:15.266321 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677848b8b7-b9vvk_dfa2a5d1-8057-4640-ae65-648fd313b181/kube-auth-proxy/0.log" Jul 18 12:38:15.414012 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:15.413986 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4kgb4_9365b5e2-a70e-42c0-8a7b-76b694c05504/payload-processing/0.log" Jul 18 12:38:16.150208 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:16.150179 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7fwbh_cf18a13e-792a-4e89-a282-1d90683d414c/serve-healthcheck-canary/0.log" Jul 18 12:38:16.771184 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:16.771147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-psvl5_9d3ecbf5-f159-4b01-a1f1-8262a78d6b81/kube-rbac-proxy/0.log" Jul 18 12:38:16.789725 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:16.789707 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-psvl5_9d3ecbf5-f159-4b01-a1f1-8262a78d6b81/exporter/0.log" Jul 18 12:38:16.808628 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:16.808609 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-psvl5_9d3ecbf5-f159-4b01-a1f1-8262a78d6b81/extractor/0.log" Jul 18 12:38:17.839477 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:17.839450 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6lsxg/perf-node-gather-daemonset-5q8zk" Jul 18 12:38:19.710516 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:19.710477 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbb59dc99-rwbbk_bc49d736-1e38-4146-bf63-53333de54985/manager/0.log" Jul 18 12:38:19.730386 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:19.730360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fbb59dc99-rwbbk_bc49d736-1e38-4146-bf63-53333de54985/copy-manifests/0.log" Jul 18 12:38:21.148388 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:21.148360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77c8bd676b-xg55x_110995e9-ae10-4681-8274-2d98ca9652fb/manager/0.log" Jul 18 12:38:21.920313 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:21.920284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677848b8b7-b9vvk_dfa2a5d1-8057-4640-ae65-648fd313b181/kube-auth-proxy/0.log" Jul 18 12:38:22.042792 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:22.042713 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4kgb4_9365b5e2-a70e-42c0-8a7b-76b694c05504/payload-processing/0.log" Jul 18 12:38:23.486029 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:23.485999 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-677848b8b7-b9vvk_dfa2a5d1-8057-4640-ae65-648fd313b181/kube-auth-proxy/0.log" Jul 18 12:38:23.605867 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:23.605837 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-4kgb4_9365b5e2-a70e-42c0-8a7b-76b694c05504/payload-processing/0.log" Jul 18 12:38:27.586450 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.586417 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5v5jq_0fe06274-ed0d-4b45-b177-61cd75f4ad1f/kube-multus/0.log" Jul 18 12:38:27.746953 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.746920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/kube-multus-additional-cni-plugins/0.log" Jul 18 12:38:27.764827 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.764798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/egress-router-binary-copy/0.log" Jul 18 12:38:27.783093 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.783068 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/cni-plugins/0.log" Jul 18 12:38:27.803260 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.803239 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/bond-cni-plugin/0.log" Jul 18 12:38:27.822364 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.822344 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/routeoverride-cni/0.log" Jul 18 12:38:27.839685 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.839631 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/whereabouts-cni-bincopy/0.log" Jul 18 12:38:27.857206 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:27.857179 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-srfjb_03b66d23-e64b-4076-9494-c10d0b7a891d/whereabouts-cni/0.log" Jul 18 12:38:28.119350 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:28.119273 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wh7n8_e785910e-4713-4292-acc8-d9d884f9d8db/network-metrics-daemon/0.log" Jul 18 12:38:28.135035 ip-10-0-130-242 kubenswrapper[2578]: I0718 12:38:28.135011 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wh7n8_e785910e-4713-4292-acc8-d9d884f9d8db/kube-rbac-proxy/0.log"