Jul 02 23:10:24.749874 ip-10-0-135-240 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:10:25.176576 ip-10-0-135-240 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:25.176576 ip-10-0-135-240 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:10:25.176576 ip-10-0-135-240 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:25.176576 ip-10-0-135-240 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:10:25.176576 ip-10-0-135-240 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:25.178765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.178640 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:10:25.186088 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186064 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.186088 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186087 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.186088 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186090 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186093 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186096 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186099 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186102 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186105 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186108 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186111 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186114 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186116 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186119 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186121 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186124 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186127 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186137 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186140 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186143 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186145 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186148 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186150 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.186193 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186153 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186155 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186159 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186162 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186166 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186170 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186173 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186176 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186179 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186182 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186185 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186187 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186191 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186194 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186197 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186200 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186203 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186206 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186209 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186211 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.186715 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186215 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186218 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186220 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186225 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186229 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186232 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186237 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186241 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186244 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186247 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186251 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186254 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186257 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186259 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186262 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186265 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186267 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186270 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186273 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.187226 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186275 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186278 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186280 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186283 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186286 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186289 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186291 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186294 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186298 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186301 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186304 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186307 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186309 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186312 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186316 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186320 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186323 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186325 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186328 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186330 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.187702 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186333 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186335 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186338 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186340 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186343 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186345 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186348 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186351 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186354 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186356 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186358 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186361 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186365 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186367 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186370 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186372 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186374 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.186378 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187142 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187149 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.188196 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187153 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187155 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187158 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187161 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187164 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187166 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187169 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187173 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187176 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187179 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187182 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187185 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187187 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187190 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187192 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187195 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187198 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187200 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187203 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187206 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.188724 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187209 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187211 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187213 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187216 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187219 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187221 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187224 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187226 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187229 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187232 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187235 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187237 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187240 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187243 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187245 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187248 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187251 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187253 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187256 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187259 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.189227 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187262 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187265 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187268 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187270 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187273 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187276 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187278 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187283 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187286 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187289 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187292 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187294 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187297 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187299 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187302 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187304 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187307 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187309 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.189788 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187314 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187318 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187321 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187325 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187328 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187331 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187334 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187336 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187339 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187342 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187345 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187347 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187351 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187353 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187356 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187359 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187362 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187364 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187367 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187371 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.190232 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187374 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187376 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187379 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187382 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187384 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187387 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187389 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187392 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187395 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187397 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187399 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187402 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187405 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187407 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187410 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187413 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187415 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187418 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187421 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.190740 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187916 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187923 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187926 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187930 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187933 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187938 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187941 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187944 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187947 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187950 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187953 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187956 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187958 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187961 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187963 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187966 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187969 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187971 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187974 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187977 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.191215 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187979 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187990 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187993 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187996 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.187999 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188001 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188004 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188007 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188009 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188012 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188015 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188017 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188020 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188023 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188025 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188028 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188031 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188033 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188036 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188038 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.191720 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188042 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188047 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188050 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188053 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188055 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188058 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188060 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188063 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188065 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188068 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188071 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188074 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188077 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188080 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188082 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188085 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188087 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188090 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188092 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188095 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.192225 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188097 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188100 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188103 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188105 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188108 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188111 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188113 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188116 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188120 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188123 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188126 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188130 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188133 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188136 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188139 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188142 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188145 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188147 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188150 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.192722 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188153 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188155 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188158 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188160 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188163 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188167 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188169 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188172 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188174 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188177 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188179 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188182 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188184 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188187 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188190 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188193 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188195 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188198 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188200 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.188203 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.193201 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188282 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188291 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188298 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188303 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188307 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188310 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188315 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188321 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188327 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188330 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188342 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188347 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188352 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188356 2558 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188359 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188362 2558 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188365 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188368 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188373 2558 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188377 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188380 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188383 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188387 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188394 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:10:25.193719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188397 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188400 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188404 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188407 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188410 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188414 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188417 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188420 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188424 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188427 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188430 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188433 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188437 2558 flags.go:64] FLAG: --enable-server="true" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188439 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188445 2558 flags.go:64] FLAG: --event-burst="100" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188448 2558 flags.go:64] FLAG: --event-qps="50" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188451 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188461 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188466 2558 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188470 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188472 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188476 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188481 2558 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188484 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188487 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:10:25.194287 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188490 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188493 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188496 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188499 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188503 2558 flags.go:64] FLAG: --feature-gates="" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188506 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188510 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188526 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188530 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188533 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188536 2558 flags.go:64] FLAG: --help="false" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188539 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188542 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188546 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188549 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188552 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188556 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188559 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188562 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188565 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188568 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188571 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188574 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188577 2558 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:10:25.194966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188580 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188588 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188601 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188605 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188607 2558 flags.go:64] FLAG: --lock-file="" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188612 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188615 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188618 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188624 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188627 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188630 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188633 2558 flags.go:64] FLAG: --logging-format="text" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188636 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188640 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188643 2558 flags.go:64] FLAG: --manifest-url="" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188645 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188651 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188654 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188658 2558 flags.go:64] FLAG: --max-pods="110" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188662 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188665 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188668 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188670 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188673 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188676 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:10:25.195625 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188679 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188687 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188690 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188693 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188696 2558 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188699 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188705 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188709 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188713 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188717 2558 flags.go:64] FLAG: --port="10250" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188720 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188723 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f6efe968ee24def5" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188727 2558 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188731 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188735 2558 flags.go:64] FLAG: --register-node="true" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188738 2558 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188742 2558 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188745 2558 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188748 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188751 2558 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188755 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188758 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188761 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188764 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188767 2558 flags.go:64] FLAG: --runonce="false" Jul 02 23:10:25.196276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188770 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188773 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188776 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188779 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188782 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188785 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188788 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188791 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188794 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188797 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188800 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188803 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188806 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188808 2558 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188811 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188818 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188821 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188825 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188829 2558 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188832 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188836 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188839 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188841 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188845 2558 flags.go:64] FLAG: --v="2" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188849 2558 flags.go:64] FLAG: --version="false" Jul 02 23:10:25.196926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188853 2558 flags.go:64] FLAG: --vmodule="" Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188857 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.188860 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189001 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189006 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189009 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189012 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189015 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189018 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189020 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189023 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189026 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189029 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189031 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189034 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189036 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189039 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189041 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189044 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189046 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.197560 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189049 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189052 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189057 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189060 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189062 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189066 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189069 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189073 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189075 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189078 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189081 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189084 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189087 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189090 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189092 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189095 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189098 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189100 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189103 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189106 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189108 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.198153 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189111 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189113 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189116 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189118 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189121 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189124 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189126 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189128 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189131 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189134 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189136 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189139 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189141 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189146 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189151 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189154 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189157 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189161 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189165 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.198731 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189169 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189172 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189174 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189177 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189180 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189184 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189187 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189190 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189193 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189195 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189198 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189200 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189203 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189206 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189210 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189213 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189216 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189218 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189221 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.199206 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189224 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189226 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189230 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189234 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189238 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189241 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189244 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189249 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189252 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189255 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189257 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189260 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189265 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189268 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189271 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189273 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189276 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189279 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189281 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.199684 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189284 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189286 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189289 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.189292 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.190358 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.197640 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.197659 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.197682 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.197687 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197747 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197753 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197757 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197760 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197763 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197766 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.200164 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197769 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197772 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197775 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197778 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197780 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197783 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197786 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197789 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197793 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197797 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197800 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197802 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197805 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197808 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197811 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197813 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197816 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197819 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197822 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197825 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.200672 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197827 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197830 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197833 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197837 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197840 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197843 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197846 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197848 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197851 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197853 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197856 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197859 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197861 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197864 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197866 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197869 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197872 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197874 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197877 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197880 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.201190 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197882 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197885 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197887 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197890 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197893 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197895 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197898 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197900 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197903 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197906 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197908 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197910 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197913 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197916 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197918 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197923 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197926 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197929 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197932 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.201710 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197934 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197938 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197942 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197945 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197948 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197951 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197954 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197957 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197960 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197963 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197965 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197968 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197970 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197973 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197976 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197978 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197981 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197983 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197986 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197988 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.202189 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197991 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197994 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197996 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.197999 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198002 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198005 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198007 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198010 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198013 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198016 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198019 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198022 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198026 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198029 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.198034 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:25.202697 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198156 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198161 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198164 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198168 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198173 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198175 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198179 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198183 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198186 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198188 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198191 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198193 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198196 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198199 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198201 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198204 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198207 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198209 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198212 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:25.203103 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198215 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198217 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198220 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198222 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198225 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198227 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198230 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198233 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198236 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198239 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198241 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198244 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198247 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198249 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198252 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198254 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198257 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198259 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198262 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198264 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:25.203588 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198267 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198269 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198271 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198274 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198276 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198279 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198282 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198284 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198287 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198289 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198292 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198294 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198297 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198299 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198302 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198304 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198307 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198310 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198312 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198315 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:25.204086 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198318 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198320 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198324 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198327 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198330 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198333 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198336 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198339 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198342 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198345 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198348 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198350 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198353 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198356 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198359 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198361 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198364 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198367 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198370 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198373 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:25.204619 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198376 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198378 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198381 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198384 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198387 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198390 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198392 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198395 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198397 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198401 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198403 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198406 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198409 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198412 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198414 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198418 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198420 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198423 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198425 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:25.205106 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:25.198428 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.198433 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.198582 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.200655 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.202834 2558 server.go:1019] "Starting client certificate rotation" Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.202941 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:10:25.205599 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.202981 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:10:25.205783 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.205717 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:10:25.211482 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.211452 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:10:25.220568 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.220543 2558 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:10:25.222977 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.222953 2558 log.go:25] "Validated CRI v1 image API" Jul 02 23:10:25.223880 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.223859 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:10:25.228802 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.228768 2558 fs.go:135] Filesystem UUIDs: map[0eb0e529-ea6e-4f19-b8e0-803f808ced56:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7a48fbee-031c-4538-9300-29d7e99e4c60:/dev/nvme0n1p3] Jul 02 23:10:25.228802 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.228791 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:10:25.234492 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.234360 2558 manager.go:216] Machine: {Timestamp:2026-07-02 23:10:25.232889556 +0000 UTC m=+0.353501764 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093151 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec284a5c89799c2cc0c87eabf2674bcc SystemUUID:ec284a5c-8979-9c2c-c0c8-7eabf2674bcc BootID:f841238e-c04c-4c93-9f8e-f38163c87f41 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:68:aa:9a:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:68:aa:9a:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:de:a2:8d:8d:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:10:25.234492 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.234486 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:10:25.234613 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.234599 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:10:25.237268 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.237248 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:10:25.237959 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.237931 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:10:25.238122 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.237964 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:10:25.239471 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.239456 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:10:25.239530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.239475 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:10:25.239530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.239484 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:10:25.240128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.240115 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:10:25.241843 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.241828 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:10:25.245389 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245372 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:10:25.245600 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245588 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:10:25.245636 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245625 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:10:25.245667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245640 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:10:25.245667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245654 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:10:25.245667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245663 2558 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:10:25.245745 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.245672 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:10:25.246056 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.246044 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:10:25.246097 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.246062 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:10:25.248679 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.248662 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:10:25.251899 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.251876 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:10:25.252014 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.251916 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:10:25.253360 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253346 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253365 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253371 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253376 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253387 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253395 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253404 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253408 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253415 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:10:25.253417 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253421 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:10:25.253707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253429 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:10:25.253707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.253435 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:10:25.254416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.254405 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:10:25.254416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.254416 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:10:25.257753 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.257736 2558 server.go:1268] "Started kubelet" Jul 02 23:10:25.257863 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.257830 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:10:25.257940 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.257892 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:10:25.257976 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.257967 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:10:25.258849 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.258824 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:10:25.258849 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.258830 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:10:25.258962 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.258851 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:10:25.259088 ip-10-0-135-240 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:10:25.260193 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.260156 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:10:25.260435 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.260420 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:10:25.265008 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.264983 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:10:25.266257 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.265055 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18be9bd130fdc5f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-07-02 23:10:25.257702903 +0000 UTC m=+0.378315158,LastTimestamp:2026-07-02 23:10:25.257702903 +0000 UTC m=+0.378315158,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}" Jul 02 23:10:25.268321 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.268298 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:10:25.268321 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.268319 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:10:25.268574 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.268557 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:25.269333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269129 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:10:25.269333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269198 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:10:25.269333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269171 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:10:25.269333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269301 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:10:25.269333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269308 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:10:25.269622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269607 2558 factory.go:55] Registering systemd factory Jul 02 23:10:25.269658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269628 2558 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:10:25.269913 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269888 2558 factory.go:153] Registering CRI-O factory Jul 02 23:10:25.269913 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269901 2558 factory.go:223] Registration of the crio container factory successfully Jul 02 23:10:25.270049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269957 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:10:25.270049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269982 2558 factory.go:103] Registering Raw factory Jul 02 23:10:25.270049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.269998 2558 manager.go:1196] Started watching for new ooms in manager Jul 02 23:10:25.270746 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.270733 2558 manager.go:349] Starting recovery of all containers Jul 02 23:10:25.273856 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.273817 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 23:10:25.273998 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.273927 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 23:10:25.282575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.282554 2558 manager.go:354] Recovery completed Jul 02 23:10:25.283476 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.283275 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wlb87" Jul 02 23:10:25.287046 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.287031 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.289342 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289325 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.289428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289356 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.289428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289366 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.289932 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289918 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:10:25.289932 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289932 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:10:25.290054 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289952 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:10:25.290054 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.289920 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.290815 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.290795 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.290891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.290828 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.290891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.290843 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.290891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.290861 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wlb87" Jul 02 23:10:25.291230 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.291219 2558 policy_none.go:49] "None policy: Start" Jul 02 23:10:25.291264 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.291236 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:10:25.291264 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.291247 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:10:25.291680 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.291606 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18be9bd132e08e00 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-07-02 23:10:25.289342464 +0000 UTC m=+0.409954672,LastTimestamp:2026-07-02 23:10:25.289342464 +0000 UTC m=+0.409954672,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}" Jul 02 23:10:25.292282 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.292270 2558 policy_none.go:47] "Start" Jul 02 23:10:25.333375 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333355 2558 manager.go:337] "Starting Device Plugin manager" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.333410 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333423 2558 server.go:87] "Starting device plugin registration server" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333762 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333774 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333902 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333984 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.333993 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.334561 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:10:25.351093 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.334605 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:25.384550 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.384446 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:10:25.385757 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.385736 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:10:25.385866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.385783 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:10:25.385866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.385815 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:10:25.385866 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.385861 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:10:25.387661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.387637 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:25.434784 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.434753 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.435691 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.435674 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.435769 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.435706 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.435769 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.435717 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.435769 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.435740 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.444236 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.444217 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.444296 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.444245 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-240.ec2.internal\": node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:25.468775 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.468747 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:25.486731 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.486693 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"] Jul 02 23:10:25.486889 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.486773 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.492890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.492869 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.493021 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.492901 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.493021 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.492911 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.494320 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.494307 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.494440 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.494426 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.494475 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.494459 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.495154 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495135 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.495154 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495137 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.495334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495179 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.495334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495193 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.495334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495161 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.495334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.495243 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.496607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.496593 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.496699 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.496627 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:25.497349 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.497332 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:25.497448 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.497354 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:25.497448 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.497364 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:25.522597 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.522564 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.527212 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.527196 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.569595 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:25.569562 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:25.570734 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.570715 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.570806 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.570742 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b12c154aed0715acfb2aca1c887480a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"3b12c154aed0715acfb2aca1c887480a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:25.570806 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:25.570764 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.723093 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:26.723056 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:26.724258 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724235 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b12c154aed0715acfb2aca1c887480a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"3b12c154aed0715acfb2aca1c887480a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.724317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.724317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724291 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.724388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724345 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.724388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724346 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b12c154aed0715acfb2aca1c887480a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"3b12c154aed0715acfb2aca1c887480a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.724388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.724367 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7a8689016a7f8195ba0457e3c7b1972e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"7a8689016a7f8195ba0457e3c7b1972e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.823893 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:26.823863 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:26.879130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.879095 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.883255 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:26.883102 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:26.924023 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:26.923993 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.024561 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.024454 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.125064 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.125022 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.225536 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.225487 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.255788 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.255751 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:10:27.256355 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.255937 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:10:27.326070 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.326043 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.346347 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.346290 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:05:26 +0000 UTC" deadline="2028-01-23 08:14:52.326401403 +0000 UTC" Jul 02 23:10:27.346542 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.346364 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13665h4m24.980068495s" Jul 02 23:10:27.391462 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.391431 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:27.420565 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:27.420527 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8689016a7f8195ba0457e3c7b1972e.slice/crio-cadf1093131b86f579ddc0d18073785e9a56038074cd6b9f64bfadb07609448a WatchSource:0}: Error finding container cadf1093131b86f579ddc0d18073785e9a56038074cd6b9f64bfadb07609448a: Status 404 returned error can't find the container with id cadf1093131b86f579ddc0d18073785e9a56038074cd6b9f64bfadb07609448a Jul 02 23:10:27.420858 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:27.420832 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b12c154aed0715acfb2aca1c887480a.slice/crio-7933aee13b32a433740c09892423f6fa0484b949fe55841ee74739ecb8cd1595 WatchSource:0}: Error finding container 7933aee13b32a433740c09892423f6fa0484b949fe55841ee74739ecb8cd1595: Status 404 returned error can't find the container with id 7933aee13b32a433740c09892423f6fa0484b949fe55841ee74739ecb8cd1595 Jul 02 23:10:27.426141 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.426095 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.426706 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.426690 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:10:27.442968 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.442923 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"3b12c154aed0715acfb2aca1c887480a","Type":"ContainerStarted","Data":"7933aee13b32a433740c09892423f6fa0484b949fe55841ee74739ecb8cd1595"} Jul 02 23:10:27.443906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.443881 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"7a8689016a7f8195ba0457e3c7b1972e","Type":"ContainerStarted","Data":"cadf1093131b86f579ddc0d18073785e9a56038074cd6b9f64bfadb07609448a"} Jul 02 23:10:27.526339 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.526306 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.626911 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:27.626819 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found" Jul 02 23:10:27.662986 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.662961 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:27.716323 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.716294 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:27.722337 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.722313 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" Jul 02 23:10:27.734837 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.734811 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:10:27.735792 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.735775 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" Jul 02 23:10:27.745480 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:27.745451 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:10:28.299332 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.299298 2558 apiserver.go:52] "Watching apiserver" Jul 02 23:10:28.306059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.306032 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:10:28.306436 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.306412 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-xmb22","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9","openshift-multus/multus-additional-cni-plugins-zwm4l","openshift-multus/network-metrics-daemon-ld4wq","openshift-network-diagnostics/network-check-target-628sv","openshift-network-operator/iptables-alerter-g55jr","openshift-ovn-kubernetes/ovnkube-node-vbp4m","kube-system/global-pull-secret-syncer-2d76z","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rkgsm","openshift-dns/node-resolver-xltcn","openshift-image-registry/node-ca-tk6cr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal","openshift-multus/multus-jwbn8"] Jul 02 23:10:28.308360 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.308335 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.309612 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.309588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.310936 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.310915 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.311217 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.311168 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:10:28.314095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.311876 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xqlnh\"" Jul 02 23:10:28.314095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.312048 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:10:28.314095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.312082 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.314095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.312116 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.314095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.312417 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:10:28.314404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314213 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.314404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314254 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.314404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.314632 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314448 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qwbg9\"" Jul 02 23:10:28.314632 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314551 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.314632 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314588 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:10:28.314789 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.314644 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:28.314789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314701 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:10:28.314789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314701 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:10:28.314789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314767 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:10:28.315052 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.314963 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.315909 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.315844 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sjgtt\"" Jul 02 23:10:28.316094 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.316066 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:28.316170 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.316147 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:28.317256 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.317243 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.318756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.318739 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.319642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.319529 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2756p\"" Jul 02 23:10:28.319642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.319571 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.319787 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.319749 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:10:28.319787 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.319780 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.320119 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.320104 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.320174 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.320159 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:28.320942 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.320923 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:10:28.321138 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.321119 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9f7ph\"" Jul 02 23:10:28.321426 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.321409 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.322616 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.322601 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:10:28.322743 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.322723 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.323596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.323579 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:10:28.323596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.323587 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rzkxm\"" Jul 02 23:10:28.323750 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.323657 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.323808 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.323784 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.324125 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.324110 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.324957 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.324942 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.325050 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.325032 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9sl5w\"" Jul 02 23:10:28.325365 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.325344 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.325451 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.325401 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.326667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.326642 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:10:28.326768 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.326692 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:10:28.326828 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.326791 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rgs2g\"" Jul 02 23:10:28.327731 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.327643 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:10:28.328123 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.327831 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:10:28.328123 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.327959 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mj9d8\"" Jul 02 23:10:28.328234 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.328172 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:10:28.331135 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-script-lib\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331222 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331147 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-daemon-config\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331222 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331170 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-multus-certs\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-host-slash\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.331328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331246 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-run\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.331328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331264 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.331328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331286 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-system-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331309 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-os-release\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-kubelet\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331355 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-lib-modules\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331384 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-tuned\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-tmp\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331432 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331500 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-ovn\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-config\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-env-overrides\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-iptables-alerter-script\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331643 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-etc-selinux\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-cnibin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-dbus\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovn-node-metrics-cert\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.331791 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331784 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-registration-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-slash\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d7nn\" (UniqueName: \"kubernetes.io/projected/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kube-api-access-7d7nn\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331896 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-socket-dir-parent\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331917 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhlpf\" (UniqueName: \"kubernetes.io/projected/a9b5cf0a-9252-4ce6-b876-32149da092bd-kube-api-access-dhlpf\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-modprobe-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.331990 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-sys\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332021 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-os-release\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332043 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8jkh\" (UniqueName: \"kubernetes.io/projected/168a5fe3-ba5c-4904-82ee-1970186c97a7-kube-api-access-z8jkh\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332065 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-cni-binary-copy\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332109 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-var-lib-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332136 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n5tt\" (UniqueName: \"kubernetes.io/projected/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-kube-api-access-9n5tt\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332159 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-log-socket\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t89s\" (UniqueName: \"kubernetes.io/projected/43d0a81a-bd52-41e9-b268-dab6d2f17a36-kube-api-access-5t89s\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332203 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7603a47-9ef4-4211-adcc-1c31975fc699-hosts-file\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332252 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7603a47-9ef4-4211-adcc-1c31975fc699-tmp-dir\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332305 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff88c77e-21fb-401f-9c2f-9c508f3270c5-agent-certs\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332347 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-kubernetes\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332377 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-multus\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332433 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-systemd-units\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332462 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysconfig\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332533 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-var-lib-kubelet\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkcf8\" (UniqueName: \"kubernetes.io/projected/94d89552-c124-4239-8ffc-00017b6745ca-kube-api-access-hkcf8\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332617 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-netns\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332640 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-hostroot\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332690 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-systemd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332715 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-node-log\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332739 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.332929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332769 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332793 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-socket-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332816 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff88c77e-21fb-401f-9c2f-9c508f3270c5-konnectivity-ca\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332838 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332870 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-systemd\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332891 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-host\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.332914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-kubelet\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333420 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zss\" (UniqueName: \"kubernetes.io/projected/f7603a47-9ef4-4211-adcc-1c31975fc699-kube-api-access-58zss\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-sys-fs\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333470 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-device-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333544 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-cnibin\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333568 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-k8s-cni-cncf-io\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333591 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-bin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333614 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-netns\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333638 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-etc-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.333712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333662 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l77m\" (UniqueName: \"kubernetes.io/projected/3f3e1d14-8e31-4a13-92a4-8264eafb9416-kube-api-access-7l77m\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333686 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-system-cni-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-conf-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333733 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-kubelet-config\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-bin\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333791 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333813 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-conf\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333858 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-etc-kubernetes\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333877 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-netd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.334309 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.333903 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.335015 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.334998 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:10:28.356484 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.356433 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dlxfb" Jul 02 23:10:28.363707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.363657 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dlxfb" Jul 02 23:10:28.423232 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.423197 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:10:28.434378 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-tuned\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.434378 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434383 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-tmp\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-ovn\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434459 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-config\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434483 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-env-overrides\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434493 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434509 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-iptables-alerter-script\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434507 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-ovn\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.434647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-etc-selinux\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434670 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434694 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-cnibin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434715 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-dbus\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434742 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovn-node-metrics-cert\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-registration-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434775 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434808 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-etc-selinux\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434814 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-slash\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434869 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d7nn\" (UniqueName: \"kubernetes.io/projected/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kube-api-access-7d7nn\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-socket-dir-parent\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434923 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhlpf\" (UniqueName: \"kubernetes.io/projected/a9b5cf0a-9252-4ce6-b876-32149da092bd-kube-api-access-dhlpf\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434932 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-dbus\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434871 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-registration-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434953 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47cnv\" (UniqueName: \"kubernetes.io/projected/d34d323b-43bd-4ffc-8818-e2d460f0f7be-kube-api-access-47cnv\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434954 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434986 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-modprobe-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.435017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435010 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-slash\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435014 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-sys\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-os-release\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435069 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8jkh\" (UniqueName: \"kubernetes.io/projected/168a5fe3-ba5c-4904-82ee-1970186c97a7-kube-api-access-z8jkh\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435094 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-cni-binary-copy\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435119 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435126 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-iptables-alerter-script\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435144 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-var-lib-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n5tt\" (UniqueName: \"kubernetes.io/projected/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-kube-api-access-9n5tt\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435187 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-config\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435198 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-log-socket\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.434810 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-cnibin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t89s\" (UniqueName: \"kubernetes.io/projected/43d0a81a-bd52-41e9-b268-dab6d2f17a36-kube-api-access-5t89s\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435245 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-os-release\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435282 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7603a47-9ef4-4211-adcc-1c31975fc699-hosts-file\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435302 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-log-socket\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7603a47-9ef4-4211-adcc-1c31975fc699-tmp-dir\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.435824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435327 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-modprobe-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435344 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-socket-dir-parent\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435366 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435381 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d34d323b-43bd-4ffc-8818-e2d460f0f7be-host\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff88c77e-21fb-401f-9c2f-9c508f3270c5-agent-certs\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7603a47-9ef4-4211-adcc-1c31975fc699-hosts-file\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435455 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-sys\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435430 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d34d323b-43bd-4ffc-8818-e2d460f0f7be-serviceca\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-kubernetes\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435530 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435576 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-multus\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435607 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-env-overrides\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435609 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-var-lib-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435638 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-multus\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-systemd-units\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435678 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7603a47-9ef4-4211-adcc-1c31975fc699-tmp-dir\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435695 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysconfig\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.436622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-var-lib-kubelet\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkcf8\" (UniqueName: \"kubernetes.io/projected/94d89552-c124-4239-8ffc-00017b6745ca-kube-api-access-hkcf8\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435815 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-netns\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435840 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-hostroot\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435866 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-systemd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435890 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-node-log\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435915 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435940 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.435963 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-socket-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff88c77e-21fb-401f-9c2f-9c508f3270c5-konnectivity-ca\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436085 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436142 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-systemd\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436162 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-netns\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.436193 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436211 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-cni-binary-copy\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-host\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436259 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-kubelet\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.437418 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436261 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-socket-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.436283 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zss\" (UniqueName: \"kubernetes.io/projected/f7603a47-9ef4-4211-adcc-1c31975fc699-kube-api-access-58zss\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436283 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-systemd\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.436324 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:28.936300294 +0000 UTC m=+3.003912837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436340 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-hostroot\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436347 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysconfig\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436366 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-host\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436310 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-systemd-units\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.436384 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:10:28.936372961 +0000 UTC m=+3.003985507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436381 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-run-systemd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436408 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-node-log\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-sys-fs\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436340 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-kubelet\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436446 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436469 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.438317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436488 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-kubernetes\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436492 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-var-lib-kubelet\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436539 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-device-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436553 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-sys-fs\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436567 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-cnibin\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436606 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a1a1d031-3374-4647-aa8b-ce2bec30dfae-device-dir\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436617 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-k8s-cni-cncf-io\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436648 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-bin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436652 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-cnibin\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436651 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-k8s-cni-cncf-io\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436674 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-netns\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436681 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-var-lib-cni-bin\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-etc-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436716 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-d\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l77m\" (UniqueName: \"kubernetes.io/projected/3f3e1d14-8e31-4a13-92a4-8264eafb9416-kube-api-access-7l77m\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436726 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-run-netns\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436759 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-system-cni-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436772 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-etc-openvswitch\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-conf-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436808 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-system-cni-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436820 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-kubelet-config\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436845 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-bin\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9467f990-5431-4897-860b-e938c0872949-kubelet-config\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-conf\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436847 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-conf-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436951 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.436953 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-bin\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437068 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff88c77e-21fb-401f-9c2f-9c508f3270c5-konnectivity-ca\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437079 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-sysctl-conf\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437094 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/168a5fe3-ba5c-4904-82ee-1970186c97a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437109 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-etc-kubernetes\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-netd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.439906 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437190 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-cni-netd\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437201 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-script-lib\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437145 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-etc-kubernetes\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437228 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-daemon-config\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437248 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437252 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-multus-certs\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437293 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-host-slash\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437319 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-run\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437349 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437354 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-host-slash\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-system-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-os-release\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437409 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-run\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437425 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-kubelet\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437451 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-lib-modules\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437458 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-system-cni-dir\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437625 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-host-run-multus-certs\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.440747 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437671 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f3e1d14-8e31-4a13-92a4-8264eafb9416-lib-modules\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437686 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9b5cf0a-9252-4ce6-b876-32149da092bd-os-release\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437711 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/43d0a81a-bd52-41e9-b268-dab6d2f17a36-host-kubelet\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a9b5cf0a-9252-4ce6-b876-32149da092bd-multus-daemon-config\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.437808 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovnkube-script-lib\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.438096 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/168a5fe3-ba5c-4904-82ee-1970186c97a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.438928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-etc-tuned\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.439228 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/43d0a81a-bd52-41e9-b268-dab6d2f17a36-ovn-node-metrics-cert\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.439307 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff88c77e-21fb-401f-9c2f-9c508f3270c5-agent-certs\") pod \"konnectivity-agent-xmb22\" (UID: \"ff88c77e-21fb-401f-9c2f-9c508f3270c5\") " pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.441615 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.439668 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f3e1d14-8e31-4a13-92a4-8264eafb9416-tmp\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.445129 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.445062 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:28.445129 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.445091 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:28.445129 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.445106 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:28.445129 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.445106 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n5tt\" (UniqueName: \"kubernetes.io/projected/f4339eb3-f89f-4c3f-88ca-1bda092c5c7e-kube-api-access-9n5tt\") pod \"iptables-alerter-g55jr\" (UID: \"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e\") " pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.445457 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.445189 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:28.945169757 +0000 UTC m=+3.012782309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:28.445636 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.445616 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d7nn\" (UniqueName: \"kubernetes.io/projected/a1a1d031-3374-4647-aa8b-ce2bec30dfae-kube-api-access-7d7nn\") pod \"aws-ebs-csi-driver-node-kvns9\" (UID: \"a1a1d031-3374-4647-aa8b-ce2bec30dfae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.446174 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.446148 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhlpf\" (UniqueName: \"kubernetes.io/projected/a9b5cf0a-9252-4ce6-b876-32149da092bd-kube-api-access-dhlpf\") pod \"multus-jwbn8\" (UID: \"a9b5cf0a-9252-4ce6-b876-32149da092bd\") " pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.446283 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.446203 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8jkh\" (UniqueName: \"kubernetes.io/projected/168a5fe3-ba5c-4904-82ee-1970186c97a7-kube-api-access-z8jkh\") pod \"multus-additional-cni-plugins-zwm4l\" (UID: \"168a5fe3-ba5c-4904-82ee-1970186c97a7\") " pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.447037 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.447011 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkcf8\" (UniqueName: \"kubernetes.io/projected/94d89552-c124-4239-8ffc-00017b6745ca-kube-api-access-hkcf8\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.447640 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.447616 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zss\" (UniqueName: \"kubernetes.io/projected/f7603a47-9ef4-4211-adcc-1c31975fc699-kube-api-access-58zss\") pod \"node-resolver-xltcn\" (UID: \"f7603a47-9ef4-4211-adcc-1c31975fc699\") " pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.450617 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.450555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l77m\" (UniqueName: \"kubernetes.io/projected/3f3e1d14-8e31-4a13-92a4-8264eafb9416-kube-api-access-7l77m\") pod \"tuned-rkgsm\" (UID: \"3f3e1d14-8e31-4a13-92a4-8264eafb9416\") " pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.450721 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.450637 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t89s\" (UniqueName: \"kubernetes.io/projected/43d0a81a-bd52-41e9-b268-dab6d2f17a36-kube-api-access-5t89s\") pod \"ovnkube-node-vbp4m\" (UID: \"43d0a81a-bd52-41e9-b268-dab6d2f17a36\") " pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.538481 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.538439 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47cnv\" (UniqueName: \"kubernetes.io/projected/d34d323b-43bd-4ffc-8818-e2d460f0f7be-kube-api-access-47cnv\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.538688 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.538507 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d34d323b-43bd-4ffc-8818-e2d460f0f7be-host\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.538688 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.538546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d34d323b-43bd-4ffc-8818-e2d460f0f7be-serviceca\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.538688 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.538663 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d34d323b-43bd-4ffc-8818-e2d460f0f7be-host\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.539060 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.539029 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d34d323b-43bd-4ffc-8818-e2d460f0f7be-serviceca\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.547640 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.547609 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47cnv\" (UniqueName: \"kubernetes.io/projected/d34d323b-43bd-4ffc-8818-e2d460f0f7be-kube-api-access-47cnv\") pod \"node-ca-tk6cr\" (UID: \"d34d323b-43bd-4ffc-8818-e2d460f0f7be\") " pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.623710 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.623616 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:28.630975 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.630945 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xltcn" Jul 02 23:10:28.639231 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.639197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" Jul 02 23:10:28.646948 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.646921 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g55jr" Jul 02 23:10:28.654217 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.654190 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jwbn8" Jul 02 23:10:28.661977 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.661953 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" Jul 02 23:10:28.670659 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.670628 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" Jul 02 23:10:28.678464 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.678441 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:28.684285 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.684267 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tk6cr" Jul 02 23:10:28.743713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.743678 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:28.941213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.941121 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:28.941213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:28.941184 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:28.941420 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.941286 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:28.941420 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.941311 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:28.941420 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.941369 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:10:29.941349138 +0000 UTC m=+4.008961712 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:28.941420 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:28.941391 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:29.941384495 +0000 UTC m=+4.008997032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:29.041886 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.041850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:29.042081 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.042037 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:29.042081 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.042063 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:29.042081 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.042076 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:29.042242 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.042148 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:30.042128822 +0000 UTC m=+4.109741378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:29.334937 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.334912 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7603a47_9ef4_4211_adcc_1c31975fc699.slice/crio-3fd272e98a5594effb1dd34fd6f029373883bbfa69a9b1dd896c4b83018902c3 WatchSource:0}: Error finding container 3fd272e98a5594effb1dd34fd6f029373883bbfa69a9b1dd896c4b83018902c3: Status 404 returned error can't find the container with id 3fd272e98a5594effb1dd34fd6f029373883bbfa69a9b1dd896c4b83018902c3 Jul 02 23:10:29.335395 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.335145 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4339eb3_f89f_4c3f_88ca_1bda092c5c7e.slice/crio-615aeb37216126761a0ff6d9062f3c15807a28faf231d950210124417bf84317 WatchSource:0}: Error finding container 615aeb37216126761a0ff6d9062f3c15807a28faf231d950210124417bf84317: Status 404 returned error can't find the container with id 615aeb37216126761a0ff6d9062f3c15807a28faf231d950210124417bf84317 Jul 02 23:10:29.341601 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.341574 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a1d031_3374_4647_aa8b_ce2bec30dfae.slice/crio-9c3138e125a4dc7f24c3a4590a4039eaae5154a644a89e0c32dd9e752603240a WatchSource:0}: Error finding container 9c3138e125a4dc7f24c3a4590a4039eaae5154a644a89e0c32dd9e752603240a: Status 404 returned error can't find the container with id 9c3138e125a4dc7f24c3a4590a4039eaae5154a644a89e0c32dd9e752603240a Jul 02 23:10:29.342510 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.342487 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34d323b_43bd_4ffc_8818_e2d460f0f7be.slice/crio-373f118581ee563a06b0cb3630fea4343ec75c6623cb031312f6a350f8770ca7 WatchSource:0}: Error finding container 373f118581ee563a06b0cb3630fea4343ec75c6623cb031312f6a350f8770ca7: Status 404 returned error can't find the container with id 373f118581ee563a06b0cb3630fea4343ec75c6623cb031312f6a350f8770ca7 Jul 02 23:10:29.343502 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.343451 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168a5fe3_ba5c_4904_82ee_1970186c97a7.slice/crio-acef7339921a692d3f82d94ea72884d41f5495e593deeb06d001d326893a5c64 WatchSource:0}: Error finding container acef7339921a692d3f82d94ea72884d41f5495e593deeb06d001d326893a5c64: Status 404 returned error can't find the container with id acef7339921a692d3f82d94ea72884d41f5495e593deeb06d001d326893a5c64 Jul 02 23:10:29.345130 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.345047 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b5cf0a_9252_4ce6_b876_32149da092bd.slice/crio-304a9132e108d85bacdd2abe7ea97ffa0ac6870eb10b86754f02a8123e84fcff WatchSource:0}: Error finding container 304a9132e108d85bacdd2abe7ea97ffa0ac6870eb10b86754f02a8123e84fcff: Status 404 returned error can't find the container with id 304a9132e108d85bacdd2abe7ea97ffa0ac6870eb10b86754f02a8123e84fcff Jul 02 23:10:29.346709 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.346615 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff88c77e_21fb_401f_9c2f_9c508f3270c5.slice/crio-9cb3a893ac9502eac8f4e767ddb055f800030fb4a248dca3ab46f49de6736d5c WatchSource:0}: Error finding container 9cb3a893ac9502eac8f4e767ddb055f800030fb4a248dca3ab46f49de6736d5c: Status 404 returned error can't find the container with id 9cb3a893ac9502eac8f4e767ddb055f800030fb4a248dca3ab46f49de6736d5c Jul 02 23:10:29.347464 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.347400 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d0a81a_bd52_41e9_b268_dab6d2f17a36.slice/crio-41b01ad980407c8a0d3c37d5542c9eca1b686929aed804ea7f9a52f4be4e054a WatchSource:0}: Error finding container 41b01ad980407c8a0d3c37d5542c9eca1b686929aed804ea7f9a52f4be4e054a: Status 404 returned error can't find the container with id 41b01ad980407c8a0d3c37d5542c9eca1b686929aed804ea7f9a52f4be4e054a Jul 02 23:10:29.349131 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:10:29.348895 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3e1d14_8e31_4a13_92a4_8264eafb9416.slice/crio-5b09ef469162d4511f73e6117638a7b1b8f9a85f649b4341c450c1cc6ea01941 WatchSource:0}: Error finding container 5b09ef469162d4511f73e6117638a7b1b8f9a85f649b4341c450c1cc6ea01941: Status 404 returned error can't find the container with id 5b09ef469162d4511f73e6117638a7b1b8f9a85f649b4341c450c1cc6ea01941 Jul 02 23:10:29.366138 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.366083 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:28 +0000 UTC" deadline="2028-02-03 02:59:18.369591083 +0000 UTC" Jul 02 23:10:29.366138 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.366137 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13923h48m49.003459746s" Jul 02 23:10:29.439897 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.439730 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:29.440077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.439786 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:29.440077 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.439988 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:29.440077 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.440036 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:29.454620 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.454586 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"3b12c154aed0715acfb2aca1c887480a","Type":"ContainerStarted","Data":"c65d4c774f1d14b82a4890caf64461d3d481b77b998ee4268ec45f821c8e6826"} Jul 02 23:10:29.456080 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.456051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" event={"ID":"3f3e1d14-8e31-4a13-92a4-8264eafb9416","Type":"ContainerStarted","Data":"5b09ef469162d4511f73e6117638a7b1b8f9a85f649b4341c450c1cc6ea01941"} Jul 02 23:10:29.462051 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.462018 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xmb22" event={"ID":"ff88c77e-21fb-401f-9c2f-9c508f3270c5","Type":"ContainerStarted","Data":"9cb3a893ac9502eac8f4e767ddb055f800030fb4a248dca3ab46f49de6736d5c"} Jul 02 23:10:29.466632 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.466605 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jwbn8" event={"ID":"a9b5cf0a-9252-4ce6-b876-32149da092bd","Type":"ContainerStarted","Data":"304a9132e108d85bacdd2abe7ea97ffa0ac6870eb10b86754f02a8123e84fcff"} Jul 02 23:10:29.469428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.469405 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerStarted","Data":"acef7339921a692d3f82d94ea72884d41f5495e593deeb06d001d326893a5c64"} Jul 02 23:10:29.470482 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.470434 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" podStartSLOduration=2.470421752 podStartE2EDuration="2.470421752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:29.470174886 +0000 UTC m=+3.537787446" watchObservedRunningTime="2026-07-02 23:10:29.470421752 +0000 UTC m=+3.538034521" Jul 02 23:10:29.470607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.470506 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tk6cr" event={"ID":"d34d323b-43bd-4ffc-8818-e2d460f0f7be","Type":"ContainerStarted","Data":"373f118581ee563a06b0cb3630fea4343ec75c6623cb031312f6a350f8770ca7"} Jul 02 23:10:29.474201 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.474171 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"41b01ad980407c8a0d3c37d5542c9eca1b686929aed804ea7f9a52f4be4e054a"} Jul 02 23:10:29.476120 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.476096 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" event={"ID":"a1a1d031-3374-4647-aa8b-ce2bec30dfae","Type":"ContainerStarted","Data":"9c3138e125a4dc7f24c3a4590a4039eaae5154a644a89e0c32dd9e752603240a"} Jul 02 23:10:29.477164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.477138 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g55jr" event={"ID":"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e","Type":"ContainerStarted","Data":"615aeb37216126761a0ff6d9062f3c15807a28faf231d950210124417bf84317"} Jul 02 23:10:29.478065 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.478044 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xltcn" event={"ID":"f7603a47-9ef4-4211-adcc-1c31975fc699","Type":"ContainerStarted","Data":"3fd272e98a5594effb1dd34fd6f029373883bbfa69a9b1dd896c4b83018902c3"} Jul 02 23:10:29.948892 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.948800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:29.948892 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:29.948875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:29.949125 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.949008 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:29.949125 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.949072 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:10:31.949052337 +0000 UTC m=+6.016664891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:29.949480 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.949460 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:29.949593 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:29.949531 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:31.949501201 +0000 UTC m=+6.017113743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:30.049506 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.049467 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:30.049732 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:30.049663 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:30.049732 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:30.049693 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:30.049732 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:30.049708 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:30.049940 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:30.049771 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:32.049752478 +0000 UTC m=+6.117365015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:30.367215 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.367093 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:28 +0000 UTC" deadline="2028-02-04 04:10:11.524342563 +0000 UTC" Jul 02 23:10:30.367215 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.367161 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13948h59m41.157185691s" Jul 02 23:10:30.446667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.446100 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:30.446667 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:30.446259 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:30.510296 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.510259 2558 generic.go:356] "Generic (PLEG): container finished" podID="7a8689016a7f8195ba0457e3c7b1972e" containerID="2ef8ab9951449c6c33927d450ae5fc10bb82439b0db9860296b399bd93c825c7" exitCode=0 Jul 02 23:10:30.511285 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:30.511254 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"7a8689016a7f8195ba0457e3c7b1972e","Type":"ContainerDied","Data":"2ef8ab9951449c6c33927d450ae5fc10bb82439b0db9860296b399bd93c825c7"} Jul 02 23:10:31.440094 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.440004 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:31.440629 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.440133 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:31.440629 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.440556 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:31.440782 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.440651 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:31.527354 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.527302 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"7a8689016a7f8195ba0457e3c7b1972e","Type":"ContainerStarted","Data":"240a9f0c78f744c88be70b3856f17ac35036e2e8baee26751df108fd1d6a83c9"} Jul 02 23:10:31.543130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.543068 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" podStartSLOduration=4.543045598 podStartE2EDuration="4.543045598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:31.542774517 +0000 UTC m=+5.610387077" watchObservedRunningTime="2026-07-02 23:10:31.543045598 +0000 UTC m=+5.610658161" Jul 02 23:10:31.966077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.965943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:31.966077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:31.966012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:31.966337 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.966166 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:31.966337 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.966167 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:31.966337 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.966230 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:10:35.966210728 +0000 UTC m=+10.033823273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:31.966337 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:31.966248 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:35.966239176 +0000 UTC m=+10.033851725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:32.067897 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:32.067184 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:32.067897 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:32.067425 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:32.067897 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:32.067444 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:32.067897 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:32.067457 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:32.067897 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:32.067531 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:36.067497579 +0000 UTC m=+10.135110138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:32.442539 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:32.441885 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:32.442539 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:32.442027 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:33.439785 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:33.439749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:33.439980 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:33.439802 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:33.439980 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:33.439889 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:33.440102 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:33.440014 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:34.439869 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:34.439832 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:34.440325 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:34.439991 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:35.439238 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:35.439163 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:35.439238 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:35.439201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:35.439500 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:35.439308 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:35.439681 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:35.439639 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:36.007350 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:36.007309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:36.007963 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:36.007407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:36.007963 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.007600 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:36.007963 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.007690 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:44.007669263 +0000 UTC m=+18.075281821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:36.008140 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.007608 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:36.008210 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.008150 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:10:44.008124526 +0000 UTC m=+18.075737066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:36.109343 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:36.108750 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:36.109343 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.108912 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:36.109343 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.108930 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:36.109343 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.108943 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:36.109343 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.109012 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:44.108984704 +0000 UTC m=+18.176597252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:36.440727 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:36.440638 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:36.441782 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:36.441745 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:37.439977 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:37.439942 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:37.440456 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:37.439941 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:37.440456 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:37.440072 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:37.440456 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:37.440222 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:38.439743 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:38.439566 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:38.439935 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:38.439748 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:39.439652 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:39.439619 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:39.440065 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:39.439619 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:39.440065 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:39.439732 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:39.440065 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:39.439835 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:40.439239 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:40.439202 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:40.439416 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:40.439357 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:41.439292 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:41.439251 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:41.439771 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:41.439396 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:41.439771 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:41.439438 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:41.439771 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:41.439525 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:42.439335 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:42.439299 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:42.439803 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:42.439444 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:43.439910 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:43.439872 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:43.440307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:43.439872 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:43.440307 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:43.439998 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:43.440307 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:43.440084 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:44.067672 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:44.067633 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:44.067857 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:44.067707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:44.067857 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.067803 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:44.067857 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.067820 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:44.068329 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.068261 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.067861426 +0000 UTC m=+34.135473968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:44.068329 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.068321 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.06830289 +0000 UTC m=+34.135915433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:44.168082 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:44.168044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:44.168251 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.168195 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:44.168251 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.168217 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:44.168251 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.168230 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:44.168471 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.168284 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.168264693 +0000 UTC m=+34.235877230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:44.439967 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:44.439890 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:44.440370 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:44.440027 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:45.439806 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:45.439768 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:45.439991 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:45.439768 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:45.439991 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:45.439887 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:45.439991 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:45.439942 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:46.440251 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:46.440220 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:46.440751 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:46.440340 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:47.439723 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:47.439681 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:47.439723 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:47.439724 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:47.439954 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:47.439834 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:47.440015 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:47.439966 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:48.440353 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.440113 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:48.440949 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:48.440473 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:48.562687 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.562648 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xmb22" event={"ID":"ff88c77e-21fb-401f-9c2f-9c508f3270c5","Type":"ContainerStarted","Data":"b3ab2fc88cd66fe82df1001bf6bb7d22ef7a5a8cbb653220cdd4bbd3521be611"} Jul 02 23:10:48.564142 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.564115 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jwbn8" event={"ID":"a9b5cf0a-9252-4ce6-b876-32149da092bd","Type":"ContainerStarted","Data":"d39dfea1f67f0f5e51edc475a205fc8250baf00a6f2b25a25b40839de5b14342"} Jul 02 23:10:48.565698 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.565673 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="b565f5d25558fe5ce0cd76300e326dfd1ddf3de43135fc5dd64bf6bfd5045dff" exitCode=0 Jul 02 23:10:48.565905 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.565739 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"b565f5d25558fe5ce0cd76300e326dfd1ddf3de43135fc5dd64bf6bfd5045dff"} Jul 02 23:10:48.567355 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.567332 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tk6cr" event={"ID":"d34d323b-43bd-4ffc-8818-e2d460f0f7be","Type":"ContainerStarted","Data":"65bd0ca371407bc4f98456d22b06da2ca1deab832ea3d0bc2bde4fff94fcdaea"} Jul 02 23:10:48.569785 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.569764 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"ebe375b9ea46aa2b19ce91dc00292cd2d01ede93e3deed68004f2c29de07a622"} Jul 02 23:10:48.569891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.569835 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"4f506e621338e4c746ddc8708b9a30a003f3c3f28d7b0a457ddd495c4875e1ce"} Jul 02 23:10:48.569891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.569886 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"56dd54b6ca1f6bf5cd27b7de1acd95a4905e6e7e379059faacce3a1351b2e5f9"} Jul 02 23:10:48.569995 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.569933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"1377908271de4aff66b8adc5ea0040a2433d05f55c811281ba6ab3a617a93e76"} Jul 02 23:10:48.571272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.571252 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" event={"ID":"a1a1d031-3374-4647-aa8b-ce2bec30dfae","Type":"ContainerStarted","Data":"5c750285babfada7016d3acbd3c3ac08fb472dfad22314db864d68df689baaeb"} Jul 02 23:10:48.573019 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.572996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xltcn" event={"ID":"f7603a47-9ef4-4211-adcc-1c31975fc699","Type":"ContainerStarted","Data":"276a9a9de2f998cf09e09ab4673f580dea2c4d90974583f32c55b102632e6fcf"} Jul 02 23:10:48.574284 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.574264 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" event={"ID":"3f3e1d14-8e31-4a13-92a4-8264eafb9416","Type":"ContainerStarted","Data":"17f1c640a2a4bc779276bdd3e90d8e13b67c2c400026e3f9a6f3d6fae72e661d"} Jul 02 23:10:48.598506 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.598456 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jwbn8" podStartSLOduration=3.809266436 podStartE2EDuration="22.598438583s" totalImagesPullingTime="18.789172147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.598197984 +0000 UTC m=+22.665810553" watchObservedRunningTime="2026-07-02 23:10:48.598438583 +0000 UTC m=+22.666051133" Jul 02 23:10:48.598709 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.598604 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xmb22" podStartSLOduration=12.155667015 podStartE2EDuration="22.598595372s" totalImagesPullingTime="10.442928357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.57821422 +0000 UTC m=+22.645826778" watchObservedRunningTime="2026-07-02 23:10:48.598595372 +0000 UTC m=+22.666207931" Jul 02 23:10:48.628008 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.626163 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rkgsm" podStartSLOduration=3.899906009 podStartE2EDuration="22.626141766s" totalImagesPullingTime="18.726235757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.625170552 +0000 UTC m=+22.692783102" watchObservedRunningTime="2026-07-02 23:10:48.626141766 +0000 UTC m=+22.693754326" Jul 02 23:10:48.640333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.640285 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tk6cr" podStartSLOduration=3.9095099429999998 podStartE2EDuration="22.640266422s" totalImagesPullingTime="18.730756479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.639719088 +0000 UTC m=+22.707331648" watchObservedRunningTime="2026-07-02 23:10:48.640266422 +0000 UTC m=+22.707878981" Jul 02 23:10:48.662758 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:48.662709 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xltcn" podStartSLOduration=3.924332773 podStartE2EDuration="22.662693286s" totalImagesPullingTime="18.738360513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.66263553 +0000 UTC m=+22.730248088" watchObservedRunningTime="2026-07-02 23:10:48.662693286 +0000 UTC m=+22.730305842" Jul 02 23:10:49.439744 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.439708 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:49.439974 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.439750 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:49.439974 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:49.439855 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:49.440070 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:49.439998 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:49.514952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.514918 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:10:49.579741 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.579705 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"3538b7a6ba9bfbed855092ccb40ef28256fbc47ba1e8a5e8e37726bffca7bc33"} Jul 02 23:10:49.579939 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.579779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"26cf956c6236f11c0966d1b28045df9aa2113a7bd35f06403ceb215cd0e8189b"} Jul 02 23:10:49.581486 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.581458 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" event={"ID":"a1a1d031-3374-4647-aa8b-ce2bec30dfae","Type":"ContainerStarted","Data":"210b46c846f25f6cc30f0c41bacdd193cca2bbb53c11fd5fe1783dec3279a4fc"} Jul 02 23:10:49.582876 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.582847 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g55jr" event={"ID":"f4339eb3-f89f-4c3f-88ca-1bda092c5c7e","Type":"ContainerStarted","Data":"f3f0267d9b2a58e4ee293022bc7712018e48b55321eee0625ffe168e3cdfa2b4"} Jul 02 23:10:49.598330 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:49.598272 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g55jr" podStartSLOduration=4.861876738 podStartE2EDuration="23.598255412s" totalImagesPullingTime="18.736378674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:49.598014021 +0000 UTC m=+23.665626590" watchObservedRunningTime="2026-07-02 23:10:49.598255412 +0000 UTC m=+23.665867970" Jul 02 23:10:50.402470 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:50.402342 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:10:49.514944997Z","UUID":"5bc6ebc7-5b64-4288-a71f-944e1bd0f97b","Handler":null,"Name":"","Endpoint":""} Jul 02 23:10:50.404084 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:50.404057 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:10:50.404206 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:50.404103 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:10:50.439424 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:50.439384 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:50.439624 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:50.439547 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:51.275288 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.275203 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:51.276119 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.276029 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:51.439341 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.439310 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:51.439554 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.439321 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:51.439554 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:51.439432 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:51.439554 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:51.439503 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:51.589647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.589347 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" event={"ID":"a1a1d031-3374-4647-aa8b-ce2bec30dfae","Type":"ContainerStarted","Data":"9c2e0a69cd94704b1824ae842392cbea8685a6fb718942afe366bd489cfe9c73"} Jul 02 23:10:51.592585 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.592555 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"86a752129c4caaac51101c15dfba119136b09046cf2710d3280ea75d0f9091d5"} Jul 02 23:10:51.592787 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.592769 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:51.593250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.593233 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xmb22" Jul 02 23:10:51.608715 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:51.608666 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kvns9" podStartSLOduration=4.763785442 podStartE2EDuration="25.608651166s" totalImagesPullingTime="20.844865724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:51.608310867 +0000 UTC m=+25.675923439" watchObservedRunningTime="2026-07-02 23:10:51.608651166 +0000 UTC m=+25.676263910" Jul 02 23:10:52.439338 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:52.439302 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:52.439879 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:52.439458 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:53.439920 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.439743 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:53.440588 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.439774 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:53.440588 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:53.440024 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:53.440588 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:53.440079 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:53.601227 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.601189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" event={"ID":"43d0a81a-bd52-41e9-b268-dab6d2f17a36","Type":"ContainerStarted","Data":"003d9d21644dbb74d9ba12dda9142192b0ba7f6b396f069e151e44ab5e57d777"} Jul 02 23:10:53.601406 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.601394 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:53.603029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.603005 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="56fe206ede0c86d5d54fa61e0616adb78f270b3f1ab2037aa33374f9f58e24f2" exitCode=0 Jul 02 23:10:53.603157 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.603086 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"56fe206ede0c86d5d54fa61e0616adb78f270b3f1ab2037aa33374f9f58e24f2"} Jul 02 23:10:53.617180 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.617154 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:53.630587 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:53.630539 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" podStartSLOduration=8.83195519 podStartE2EDuration="27.630506935s" totalImagesPullingTime="18.798551745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:53.629971961 +0000 UTC m=+27.697584520" watchObservedRunningTime="2026-07-02 23:10:53.630506935 +0000 UTC m=+27.698119493" Jul 02 23:10:54.439838 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:54.439809 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:54.440090 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:54.439940 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:54.606054 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:54.606026 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:54.606271 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:54.606095 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:54.621188 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:54.621165 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:10:55.439144 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.439055 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:55.439357 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.439057 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:55.439357 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:55.439168 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:55.439357 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:55.439247 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:55.523567 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.523534 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2d76z"] Jul 02 23:10:55.526296 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.526265 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-628sv"] Jul 02 23:10:55.526933 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.526906 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ld4wq"] Jul 02 23:10:55.527046 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.527033 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:55.527167 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:55.527143 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:55.609720 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.609685 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="80b3c7d7c764f1e8de4497412311ecdcef1f98ad7b25606be3b154abab41e7e9" exitCode=0 Jul 02 23:10:55.609895 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.609773 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"80b3c7d7c764f1e8de4497412311ecdcef1f98ad7b25606be3b154abab41e7e9"} Jul 02 23:10:55.609960 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.609916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:55.609960 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:55.609938 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:55.610042 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:55.610022 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:55.610121 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:55.610094 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:57.439748 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:57.439509 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:57.440222 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:57.439510 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:57.440222 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:57.439796 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:57.440222 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:57.439564 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:57.440222 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:57.439853 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:10:57.440222 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:57.439921 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:57.618160 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:57.618123 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="f3aa94ede4f82f61be3fedf465f21c82f815e56ebf3dc001812c191f6fa966ab" exitCode=0 Jul 02 23:10:57.618333 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:57.618196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"f3aa94ede4f82f61be3fedf465f21c82f815e56ebf3dc001812c191f6fa966ab"} Jul 02 23:10:59.439752 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:59.439715 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:10:59.439752 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:59.439752 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:10:59.440305 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:10:59.439757 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:10:59.440305 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:59.439849 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-628sv" podUID="61570ae8-c673-4e96-84cc-dcec5c8c5525" Jul 02 23:10:59.440305 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:59.439946 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4wq" podUID="94d89552-c124-4239-8ffc-00017b6745ca" Jul 02 23:10:59.440305 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:10:59.440048 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2d76z" podUID="9467f990-5431-4897-860b-e938c0872949" Jul 02 23:11:00.081866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:00.081825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:11:00.082069 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:00.081933 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:11:00.082069 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.082003 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:11:00.082069 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.082038 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:11:00.082202 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.082081 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs podName:94d89552-c124-4239-8ffc-00017b6745ca nodeName:}" failed. No retries permitted until 2026-07-02 23:11:32.082058754 +0000 UTC m=+66.149671313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs") pod "network-metrics-daemon-ld4wq" (UID: "94d89552-c124-4239-8ffc-00017b6745ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:11:00.082202 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.082103 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret podName:9467f990-5431-4897-860b-e938c0872949 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:32.082092672 +0000 UTC m=+66.149705223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret") pod "global-pull-secret-syncer-2d76z" (UID: "9467f990-5431-4897-860b-e938c0872949") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:11:00.182943 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:00.182905 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:00.183131 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.183070 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:11:00.183131 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.183088 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:11:00.183131 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.183098 2558 projected.go:196] Error preparing data for projected volume kube-api-access-wlm2n for pod openshift-network-diagnostics/network-check-target-628sv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:11:00.183282 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:00.183150 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n podName:61570ae8-c673-4e96-84cc-dcec5c8c5525 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:32.183133755 +0000 UTC m=+66.250746312 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wlm2n" (UniqueName: "kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n") pod "network-check-target-628sv" (UID: "61570ae8-c673-4e96-84cc-dcec5c8c5525") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:11:01.144214 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.144138 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeReady" Jul 02 23:11:01.144719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.144293 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:11:01.206448 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.206399 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m"] Jul 02 23:11:01.209127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.209094 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-q62ks"] Jul 02 23:11:01.209277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.209245 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.210760 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.210740 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv"] Jul 02 23:11:01.211098 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.210866 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.212673 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.212646 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr"] Jul 02 23:11:01.212804 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.212771 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.214445 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.214424 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj"] Jul 02 23:11:01.214581 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.214548 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" Jul 02 23:11:01.216139 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.216120 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:11:01.216260 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.216246 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.218266 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.218098 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.224231 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.224063 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.224231 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.224068 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 02 23:11:01.224231 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.224109 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.224566 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.224443 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 23:11:01.225355 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225334 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.225475 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225344 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.225475 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225389 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 02 23:11:01.225475 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225445 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225818 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225853 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225928 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.225978 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fgz6z\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226078 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kcsp9\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226089 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226123 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.226216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226167 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226226 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226280 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226349 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xdfpx\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226386 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226401 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226429 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226350 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nz7lp\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226354 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4mjl7\"" Jul 02 23:11:01.226774 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.226562 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 23:11:01.227176 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.227099 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 23:11:01.227346 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.227261 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jqj9h\"" Jul 02 23:11:01.231692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.231358 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 02 23:11:01.232924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.232900 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m"] Jul 02 23:11:01.240237 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.240215 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 23:11:01.244586 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.244560 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj"] Jul 02 23:11:01.247312 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.247288 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:11:01.248488 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.248271 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv"] Jul 02 23:11:01.249793 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.249407 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr"] Jul 02 23:11:01.264933 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.264904 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-q62ks"] Jul 02 23:11:01.290917 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.290881 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.290917 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.290923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.291194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.290948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzxhs\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.291194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291029 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.291194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291071 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.291194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7zmx\" (UniqueName: \"kubernetes.io/projected/d5c7e9b5-df90-45af-bedd-66786f0fed2a-kube-api-access-r7zmx\") pod \"network-check-source-989f8b8dc-pdlhr\" (UID: \"d5c7e9b5-df90-45af-bedd-66786f0fed2a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" Jul 02 23:11:01.291194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291201 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-trusted-ca\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291223 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af98f986-4915-48d1-80fa-4a8a11b99dc1-serving-cert\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291246 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb41a52-c6bd-495a-9315-1173670e2b58-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291293 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-config\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291323 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slfs9\" (UniqueName: \"kubernetes.io/projected/af98f986-4915-48d1-80fa-4a8a11b99dc1-kube-api-access-slfs9\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291348 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291373 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkzrv\" (UniqueName: \"kubernetes.io/projected/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-kube-api-access-bkzrv\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291398 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg88d\" (UniqueName: \"kubernetes.io/projected/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-kube-api-access-fg88d\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.291499 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxk89\" (UniqueName: \"kubernetes.io/projected/9fb41a52-c6bd-495a-9315-1173670e2b58-kube-api-access-kxk89\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.292039 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291563 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-config\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.292039 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291593 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.292039 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.292039 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.291646 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb41a52-c6bd-495a-9315-1173670e2b58-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.303756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.303722 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rmmhq"] Jul 02 23:11:01.305866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.305844 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.309604 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.309577 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:11:01.309757 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.309577 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:11:01.309757 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.309622 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jlfnl\"" Jul 02 23:11:01.324756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.324725 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rmmhq"] Jul 02 23:11:01.346073 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.346039 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-784c48b49-hfwdq"] Jul 02 23:11:01.348209 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.348174 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.349434 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.349399 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-w4sdf"] Jul 02 23:11:01.351014 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.350983 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 02 23:11:01.351258 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.351238 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 02 23:11:01.351359 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.351318 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pw8md\"" Jul 02 23:11:01.351563 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.351543 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kz2k8"] Jul 02 23:11:01.351654 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.351617 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.353306 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.353286 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn"] Jul 02 23:11:01.353422 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.353394 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.355073 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.355057 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.356740 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.356724 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.356867 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.356843 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 02 23:11:01.356958 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.356889 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.357227 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.357206 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 02 23:11:01.357328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.357243 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rhkvs\"" Jul 02 23:11:01.357328 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.357318 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 02 23:11:01.357575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.357487 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.360490 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.360442 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:11:01.360642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.360529 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.361038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.360781 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 02 23:11:01.361038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.360798 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.361038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361003 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 02 23:11:01.361229 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361056 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.361229 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361128 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.361229 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361154 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s62t9\"" Jul 02 23:11:01.361433 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361419 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x9d77\"" Jul 02 23:11:01.361733 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.361715 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.364768 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.364746 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-784c48b49-hfwdq"] Jul 02 23:11:01.368926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.368900 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 02 23:11:01.374117 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.373941 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-w4sdf"] Jul 02 23:11:01.376642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.376619 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kz2k8"] Jul 02 23:11:01.378792 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.378768 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh"] Jul 02 23:11:01.381029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.381012 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" Jul 02 23:11:01.385249 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.385225 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bs46k\"" Jul 02 23:11:01.385666 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.385616 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:11:01.385981 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.385960 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 23:11:01.392931 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.392907 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slfs9\" (UniqueName: \"kubernetes.io/projected/af98f986-4915-48d1-80fa-4a8a11b99dc1-kube-api-access-slfs9\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.393068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.392955 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.393068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.392995 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkzrv\" (UniqueName: \"kubernetes.io/projected/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-kube-api-access-bkzrv\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.393068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.393068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg88d\" (UniqueName: \"kubernetes.io/projected/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-kube-api-access-fg88d\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxk89\" (UniqueName: \"kubernetes.io/projected/9fb41a52-c6bd-495a-9315-1173670e2b58-kube-api-access-kxk89\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.393098 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393118 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-config\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393144 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.393167 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:01.893144944 +0000 UTC m=+35.960757495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb41a52-c6bd-495a-9315-1173670e2b58-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393267 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b406076e-94af-44ac-97c0-e2eba90effa4-config-volume\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393314 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzxhs\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393389 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vhn7\" (UniqueName: \"kubernetes.io/projected/b406076e-94af-44ac-97c0-e2eba90effa4-kube-api-access-9vhn7\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393446 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393537 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7zmx\" (UniqueName: \"kubernetes.io/projected/d5c7e9b5-df90-45af-bedd-66786f0fed2a-kube-api-access-r7zmx\") pod \"network-check-source-989f8b8dc-pdlhr\" (UID: \"d5c7e9b5-df90-45af-bedd-66786f0fed2a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393558 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-trusted-ca\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af98f986-4915-48d1-80fa-4a8a11b99dc1-serving-cert\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393657 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb41a52-c6bd-495a-9315-1173670e2b58-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-config\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393715 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393741 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b406076e-94af-44ac-97c0-e2eba90effa4-tmp-dir\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.393861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.393837 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-config\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.394376 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.394239 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.395211 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.394477 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-config\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.395211 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.395083 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.396289 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.396114 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:01.396289 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.396133 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:01.396289 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.396184 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:01.89616706 +0000 UTC m=+35.963779603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:01.397442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.396953 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.397442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.397235 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh"] Jul 02 23:11:01.397442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.397317 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn"] Jul 02 23:11:01.397909 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.397885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb41a52-c6bd-495a-9315-1173670e2b58-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.398152 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.398127 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.398611 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.398555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.398705 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.398642 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af98f986-4915-48d1-80fa-4a8a11b99dc1-trusted-ca\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.398818 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.398793 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb41a52-c6bd-495a-9315-1173670e2b58-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.399572 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.399551 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2"] Jul 02 23:11:01.400026 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.400005 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.401497 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.401435 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.403306 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.403283 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af98f986-4915-48d1-80fa-4a8a11b99dc1-serving-cert\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.403627 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.403607 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.407154 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.407126 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 23:11:01.407383 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.407359 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 23:11:01.407462 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.407365 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l4t2n\"" Jul 02 23:11:01.408243 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.408212 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxk89\" (UniqueName: \"kubernetes.io/projected/9fb41a52-c6bd-495a-9315-1173670e2b58-kube-api-access-kxk89\") pod \"kube-storage-version-migrator-operator-846bf668f7-m5qfj\" (UID: \"9fb41a52-c6bd-495a-9315-1173670e2b58\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.408682 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.408661 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slfs9\" (UniqueName: \"kubernetes.io/projected/af98f986-4915-48d1-80fa-4a8a11b99dc1-kube-api-access-slfs9\") pod \"console-operator-68b757fb67-q62ks\" (UID: \"af98f986-4915-48d1-80fa-4a8a11b99dc1\") " pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.408824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.408757 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzxhs\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.410304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.410282 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7zmx\" (UniqueName: \"kubernetes.io/projected/d5c7e9b5-df90-45af-bedd-66786f0fed2a-kube-api-access-r7zmx\") pod \"network-check-source-989f8b8dc-pdlhr\" (UID: \"d5c7e9b5-df90-45af-bedd-66786f0fed2a\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" Jul 02 23:11:01.411339 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.411320 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkzrv\" (UniqueName: \"kubernetes.io/projected/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-kube-api-access-bkzrv\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.419857 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.419802 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.420365 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.420297 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2"] Jul 02 23:11:01.420785 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.420715 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg88d\" (UniqueName: \"kubernetes.io/projected/a56a6e4c-8e55-4175-8d28-b43ffe93c9b6-kube-api-access-fg88d\") pod \"service-ca-operator-5d46d9ffd4-56ntv\" (UID: \"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.439164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.439108 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:11:01.439164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.439133 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:11:01.439164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.439133 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:01.443182 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.443158 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gmk5n\"" Jul 02 23:11:01.443532 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.443492 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:11:01.443646 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.443571 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:11:01.443710 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.443661 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-prtl8\"" Jul 02 23:11:01.494720 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.494901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m64m\" (UniqueName: \"kubernetes.io/projected/116d4405-27af-41ac-a5a3-b13cffd78d7b-kube-api-access-4m64m\") pod \"volume-data-source-validator-5d456fdbcd-dk7wh\" (UID: \"116d4405-27af-41ac-a5a3-b13cffd78d7b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" Jul 02 23:11:01.494901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494766 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62a2b086-a180-428e-abce-6516c0c1d001-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.494901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494866 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494921 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.495029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494952 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-stats-auth\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.495029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494969 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d23b8783-dbea-4420-98a5-85a56ff2e7a9-cert\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.495029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.494984 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.495029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b406076e-94af-44ac-97c0-e2eba90effa4-config-volume\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrv75\" (UniqueName: \"kubernetes.io/projected/da583992-cc89-437b-a199-67b2a476ad7a-kube-api-access-zrv75\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495095 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vhn7\" (UniqueName: \"kubernetes.io/projected/b406076e-94af-44ac-97c0-e2eba90effa4-kube-api-access-9vhn7\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-snapshots\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495201 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b406076e-94af-44ac-97c0-e2eba90effa4-tmp-dir\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.495272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k9j4\" (UniqueName: \"kubernetes.io/projected/a9a9fb53-6114-4a00-9fef-138d8ee74f63-kube-api-access-4k9j4\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4zm5\" (UniqueName: \"kubernetes.io/projected/d23b8783-dbea-4420-98a5-85a56ff2e7a9-kube-api-access-w4zm5\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495365 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-default-certificate\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.495377 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.495448 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:01.995426897 +0000 UTC m=+36.063039438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495484 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-tmp\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xh2g\" (UniqueName: \"kubernetes.io/projected/f51c65fa-18ca-426b-9beb-c15a20ce99e5-kube-api-access-7xh2g\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.495607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da583992-cc89-437b-a199-67b2a476ad7a-serving-cert\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.495911 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495658 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b406076e-94af-44ac-97c0-e2eba90effa4-tmp-dir\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.495911 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.495689 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b406076e-94af-44ac-97c0-e2eba90effa4-config-volume\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.506023 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.505994 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vhn7\" (UniqueName: \"kubernetes.io/projected/b406076e-94af-44ac-97c0-e2eba90effa4-kube-api-access-9vhn7\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:01.531725 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.531690 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" Jul 02 23:11:01.538059 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.538035 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:01.547434 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.547405 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" Jul 02 23:11:01.554370 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.554333 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" Jul 02 23:11:01.596663 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.596247 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.596861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.596705 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m64m\" (UniqueName: \"kubernetes.io/projected/116d4405-27af-41ac-a5a3-b13cffd78d7b-kube-api-access-4m64m\") pod \"volume-data-source-validator-5d456fdbcd-dk7wh\" (UID: \"116d4405-27af-41ac-a5a3-b13cffd78d7b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" Jul 02 23:11:01.596861 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.596733 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:01.596861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.596747 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62a2b086-a180-428e-abce-6516c0c1d001-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.596861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.596791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.596861 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.596837 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.096815247 +0000 UTC m=+36.164427798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:01.597379 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.597479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-stats-auth\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.597479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d23b8783-dbea-4420-98a5-85a56ff2e7a9-cert\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.597479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.597479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597455 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.597479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597473 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597491 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/62a2b086-a180-428e-abce-6516c0c1d001-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrv75\" (UniqueName: \"kubernetes.io/projected/da583992-cc89-437b-a199-67b2a476ad7a-kube-api-access-zrv75\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-snapshots\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597647 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k9j4\" (UniqueName: \"kubernetes.io/projected/a9a9fb53-6114-4a00-9fef-138d8ee74f63-kube-api-access-4k9j4\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.597661 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597682 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4zm5\" (UniqueName: \"kubernetes.io/projected/d23b8783-dbea-4420-98a5-85a56ff2e7a9-kube-api-access-w4zm5\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597713 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-default-certificate\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.597765 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.597735 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.097711665 +0000 UTC m=+36.165324204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:01.598151 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.598151 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597806 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-tmp\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.598151 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597874 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xh2g\" (UniqueName: \"kubernetes.io/projected/f51c65fa-18ca-426b-9beb-c15a20ce99e5-kube-api-access-7xh2g\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.598151 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.597910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da583992-cc89-437b-a199-67b2a476ad7a-serving-cert\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.598151 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.597945 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.097929105 +0000 UTC m=+36.165541672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:01.598949 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.598453 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-tmp\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.598949 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.598583 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:01.598949 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.598651 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.098635601 +0000 UTC m=+36.166248146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:01.599905 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.599858 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da583992-cc89-437b-a199-67b2a476ad7a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.601552 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.600540 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/da583992-cc89-437b-a199-67b2a476ad7a-snapshots\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.601552 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.600607 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d23b8783-dbea-4420-98a5-85a56ff2e7a9-cert\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.601552 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.601087 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-default-certificate\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.602320 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.602051 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-stats-auth\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.603852 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.603827 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da583992-cc89-437b-a199-67b2a476ad7a-serving-cert\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.608240 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.608203 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xh2g\" (UniqueName: \"kubernetes.io/projected/f51c65fa-18ca-426b-9beb-c15a20ce99e5-kube-api-access-7xh2g\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:01.608240 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.608235 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m64m\" (UniqueName: \"kubernetes.io/projected/116d4405-27af-41ac-a5a3-b13cffd78d7b-kube-api-access-4m64m\") pod \"volume-data-source-validator-5d456fdbcd-dk7wh\" (UID: \"116d4405-27af-41ac-a5a3-b13cffd78d7b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" Jul 02 23:11:01.608572 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.608509 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4zm5\" (UniqueName: \"kubernetes.io/projected/d23b8783-dbea-4420-98a5-85a56ff2e7a9-kube-api-access-w4zm5\") pod \"ingress-canary-kz2k8\" (UID: \"d23b8783-dbea-4420-98a5-85a56ff2e7a9\") " pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.608670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.608590 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrv75\" (UniqueName: \"kubernetes.io/projected/da583992-cc89-437b-a199-67b2a476ad7a-kube-api-access-zrv75\") pod \"insights-operator-5c847dcfc4-w4sdf\" (UID: \"da583992-cc89-437b-a199-67b2a476ad7a\") " pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.608729 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.608716 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k9j4\" (UniqueName: \"kubernetes.io/projected/a9a9fb53-6114-4a00-9fef-138d8ee74f63-kube-api-access-4k9j4\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:01.673741 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.673656 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" Jul 02 23:11:01.681271 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.681240 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kz2k8" Jul 02 23:11:01.695239 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.695207 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" Jul 02 23:11:01.902711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.902242 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:01.902711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:01.902348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:01.902711 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.902638 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:01.902711 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.902710 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.902688176 +0000 UTC m=+36.970300732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:01.903287 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.903184 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:01.903287 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.903209 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:01.903287 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:01.903261 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:02.903242195 +0000 UTC m=+36.970854748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:02.005185 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.004311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:02.005185 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.004702 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:02.005185 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.004771 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:03.004750287 +0000 UTC m=+37.072362847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.105639 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.105740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.105795 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.105823 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.105883 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.105930 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.105958 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:03.105936242 +0000 UTC m=+37.173548792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.105990 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:03.105972842 +0000 UTC m=+37.173585409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.105996 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.106040 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:03.106026589 +0000 UTC m=+37.173639126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:02.106087 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.106056 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:03.10604605 +0000 UTC m=+37.173658616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:02.130251 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.130126 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr"] Jul 02 23:11:02.137282 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.137232 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c7e9b5_df90_45af_bedd_66786f0fed2a.slice/crio-0cf30913cb52f73e517921c034a88c5acf6dbb8b49abfef601c34104b2f0415f WatchSource:0}: Error finding container 0cf30913cb52f73e517921c034a88c5acf6dbb8b49abfef601c34104b2f0415f: Status 404 returned error can't find the container with id 0cf30913cb52f73e517921c034a88c5acf6dbb8b49abfef601c34104b2f0415f Jul 02 23:11:02.139232 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.138766 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh"] Jul 02 23:11:02.139618 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.139557 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj"] Jul 02 23:11:02.146078 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.146042 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kz2k8"] Jul 02 23:11:02.148501 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.148480 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv"] Jul 02 23:11:02.153060 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.153035 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-q62ks"] Jul 02 23:11:02.158572 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.158545 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-w4sdf"] Jul 02 23:11:02.161733 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.161708 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb41a52_c6bd_495a_9315_1173670e2b58.slice/crio-835c826af2e344bb87577b449cb609a7519d47fb2282c9cebf15a1d0863afdc9 WatchSource:0}: Error finding container 835c826af2e344bb87577b449cb609a7519d47fb2282c9cebf15a1d0863afdc9: Status 404 returned error can't find the container with id 835c826af2e344bb87577b449cb609a7519d47fb2282c9cebf15a1d0863afdc9 Jul 02 23:11:02.162304 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.162276 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23b8783_dbea_4420_98a5_85a56ff2e7a9.slice/crio-07dd1f3e59631523d2f7ac566d3a54d027134038e2f9180e5cc4ab1a04e805b3 WatchSource:0}: Error finding container 07dd1f3e59631523d2f7ac566d3a54d027134038e2f9180e5cc4ab1a04e805b3: Status 404 returned error can't find the container with id 07dd1f3e59631523d2f7ac566d3a54d027134038e2f9180e5cc4ab1a04e805b3 Jul 02 23:11:02.163564 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.163544 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116d4405_27af_41ac_a5a3_b13cffd78d7b.slice/crio-d426897a8b8c56fcfb12a11de61dfdeda6aa65357bb1e1d94af9aeba7b59be6e WatchSource:0}: Error finding container d426897a8b8c56fcfb12a11de61dfdeda6aa65357bb1e1d94af9aeba7b59be6e: Status 404 returned error can't find the container with id d426897a8b8c56fcfb12a11de61dfdeda6aa65357bb1e1d94af9aeba7b59be6e Jul 02 23:11:02.164923 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.164455 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56a6e4c_8e55_4175_8d28_b43ffe93c9b6.slice/crio-8154bbb957518e822b75d04e4e504cbc9a01d7b9f4e7b104f050db145863d750 WatchSource:0}: Error finding container 8154bbb957518e822b75d04e4e504cbc9a01d7b9f4e7b104f050db145863d750: Status 404 returned error can't find the container with id 8154bbb957518e822b75d04e4e504cbc9a01d7b9f4e7b104f050db145863d750 Jul 02 23:11:02.165835 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.165413 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda583992_cc89_437b_a199_67b2a476ad7a.slice/crio-3d5c9d29e02d48366c227c407383f520c777fb473eb0e8804ae069ae7d95b025 WatchSource:0}: Error finding container 3d5c9d29e02d48366c227c407383f520c777fb473eb0e8804ae069ae7d95b025: Status 404 returned error can't find the container with id 3d5c9d29e02d48366c227c407383f520c777fb473eb0e8804ae069ae7d95b025 Jul 02 23:11:02.166398 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:02.166370 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf98f986_4915_48d1_80fa_4a8a11b99dc1.slice/crio-99db34caaeee6c45559744c6253b1d62bb034ae64d7da26b6f1c3f19e3a478cf WatchSource:0}: Error finding container 99db34caaeee6c45559744c6253b1d62bb034ae64d7da26b6f1c3f19e3a478cf: Status 404 returned error can't find the container with id 99db34caaeee6c45559744c6253b1d62bb034ae64d7da26b6f1c3f19e3a478cf Jul 02 23:11:02.632886 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.632793 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" event={"ID":"d5c7e9b5-df90-45af-bedd-66786f0fed2a","Type":"ContainerStarted","Data":"0cf30913cb52f73e517921c034a88c5acf6dbb8b49abfef601c34104b2f0415f"} Jul 02 23:11:02.638360 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.637768 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="bc38c74bb4362eaffec7382a9dfa8a59849fb180803340097409903ab04d9376" exitCode=0 Jul 02 23:11:02.638360 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.638129 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"bc38c74bb4362eaffec7382a9dfa8a59849fb180803340097409903ab04d9376"} Jul 02 23:11:02.639912 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.639845 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" event={"ID":"af98f986-4915-48d1-80fa-4a8a11b99dc1","Type":"ContainerStarted","Data":"99db34caaeee6c45559744c6253b1d62bb034ae64d7da26b6f1c3f19e3a478cf"} Jul 02 23:11:02.641565 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.641479 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" event={"ID":"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6","Type":"ContainerStarted","Data":"8154bbb957518e822b75d04e4e504cbc9a01d7b9f4e7b104f050db145863d750"} Jul 02 23:11:02.643677 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.643626 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" event={"ID":"9fb41a52-c6bd-495a-9315-1173670e2b58","Type":"ContainerStarted","Data":"835c826af2e344bb87577b449cb609a7519d47fb2282c9cebf15a1d0863afdc9"} Jul 02 23:11:02.644795 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.644769 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" event={"ID":"da583992-cc89-437b-a199-67b2a476ad7a","Type":"ContainerStarted","Data":"3d5c9d29e02d48366c227c407383f520c777fb473eb0e8804ae069ae7d95b025"} Jul 02 23:11:02.648085 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.648057 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" event={"ID":"116d4405-27af-41ac-a5a3-b13cffd78d7b","Type":"ContainerStarted","Data":"d426897a8b8c56fcfb12a11de61dfdeda6aa65357bb1e1d94af9aeba7b59be6e"} Jul 02 23:11:02.651489 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.651422 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kz2k8" event={"ID":"d23b8783-dbea-4420-98a5-85a56ff2e7a9","Type":"ContainerStarted","Data":"07dd1f3e59631523d2f7ac566d3a54d027134038e2f9180e5cc4ab1a04e805b3"} Jul 02 23:11:02.914784 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.914746 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:02.914957 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:02.914934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:02.915549 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.915101 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:02.915549 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.915172 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.915153775 +0000 UTC m=+38.982766320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:02.915549 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.915196 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:02.915549 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.915209 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:02.915549 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:02.915245 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.915233779 +0000 UTC m=+38.982846331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:03.016277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.016200 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:03.016452 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.016348 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:03.016452 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.016409 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:05.016389946 +0000 UTC m=+39.084002484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:03.116675 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.116635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:03.116857 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.116736 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:03.116857 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.116791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:03.116857 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.116819 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:03.117069 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.116970 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:03.117069 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117042 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:05.117023198 +0000 UTC m=+39.184635739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:03.117328 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117303 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:03.117406 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117337 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:05.117319464 +0000 UTC m=+39.184932013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:03.117406 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117371 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:05.117352316 +0000 UTC m=+39.184964873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:03.117406 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117303 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:03.117635 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:03.117430 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:05.117404828 +0000 UTC m=+39.185017368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:03.667086 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.666038 2558 generic.go:356] "Generic (PLEG): container finished" podID="168a5fe3-ba5c-4904-82ee-1970186c97a7" containerID="c1c6c6ac9f5b9a832546c622e8b6fe762e1aac695b465282c2f5aefdb2218d6d" exitCode=0 Jul 02 23:11:03.667086 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:03.666113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerDied","Data":"c1c6c6ac9f5b9a832546c622e8b6fe762e1aac695b465282c2f5aefdb2218d6d"} Jul 02 23:11:04.674614 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:04.674308 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" event={"ID":"168a5fe3-ba5c-4904-82ee-1970186c97a7","Type":"ContainerStarted","Data":"e46fc4329b3a8a784688c8327aa8c106949a4cdfdaee24b2b8263989ffcec51f"} Jul 02 23:11:04.702216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:04.702151 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zwm4l" podStartSLOduration=8.59565849 podStartE2EDuration="38.702128656s" totalImagesPullingTime="30.106470166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:04.69917102 +0000 UTC m=+38.766783591" watchObservedRunningTime="2026-07-02 23:11:04.702128656 +0000 UTC m=+38.769741215" Jul 02 23:11:04.936755 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:04.936566 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:04.936755 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:04.936673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:04.936969 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:04.936788 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:04.936969 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:04.936882 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:04.936969 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:04.936896 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:04.936969 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:04.936939 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:08.936917233 +0000 UTC m=+43.004529773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:04.936969 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:04.936959 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:08.936949627 +0000 UTC m=+43.004562170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:05.037860 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:05.037821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:05.038064 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.038009 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:05.038125 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.038088 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:09.038066213 +0000 UTC m=+43.105678774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:05.138824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:05.138788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:05.138982 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:05.138857 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:05.138982 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:05.138894 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:05.138982 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:05.138912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.138977 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139016 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139023 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139061 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:09.139040045 +0000 UTC m=+43.206652602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139083 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:09.139071752 +0000 UTC m=+43.206684290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139098 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:09.139090005 +0000 UTC m=+43.206702547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:05.139154 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:05.139113 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:09.139105838 +0000 UTC m=+43.206718376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:08.976117 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:08.976069 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:08.976188 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:08.976251 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:08.976313 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:08.976328 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:08.976330 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:16.976309528 +0000 UTC m=+51.043922078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:08.976925 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:08.976375 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:16.976361884 +0000 UTC m=+51.043974434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:09.076889 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:09.076849 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:09.077061 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.077027 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:09.077112 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.077099 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:17.077082046 +0000 UTC m=+51.144694588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:09.177673 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:09.177636 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:09.177789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:09.177704 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:09.177789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:09.177750 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:09.177789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:09.177768 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:09.177888 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177822 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:09.177888 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177856 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:09.177888 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177871 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:09.177976 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177892 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:17.177872466 +0000 UTC m=+51.245485008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:09.177976 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177915 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:17.177906675 +0000 UTC m=+51.245519224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:09.177976 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177928 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:17.177921492 +0000 UTC m=+51.245534031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:09.177976 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:09.177944 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:17.17793584 +0000 UTC m=+51.245548382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:10.693755 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.693729 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/0.log" Jul 02 23:11:10.694235 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.693770 2558 generic.go:356] "Generic (PLEG): container finished" podID="af98f986-4915-48d1-80fa-4a8a11b99dc1" containerID="26593a72f482d6de7be208375eabbaf439f088c6c533b02adce4c26e6f5cc008" exitCode=255 Jul 02 23:11:10.694235 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.693849 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" event={"ID":"af98f986-4915-48d1-80fa-4a8a11b99dc1","Type":"ContainerDied","Data":"26593a72f482d6de7be208375eabbaf439f088c6c533b02adce4c26e6f5cc008"} Jul 02 23:11:10.694235 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.694121 2558 scope.go:117] "RemoveContainer" containerID="26593a72f482d6de7be208375eabbaf439f088c6c533b02adce4c26e6f5cc008" Jul 02 23:11:10.695636 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.695610 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" event={"ID":"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6","Type":"ContainerStarted","Data":"eb5d5c2d836780f079b01a9246bb436c40166d33afc17c08f4b62c3f3e5a6ce8"} Jul 02 23:11:10.697336 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.697306 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" event={"ID":"9fb41a52-c6bd-495a-9315-1173670e2b58","Type":"ContainerStarted","Data":"1ab4391452fdadfac2ab7587f0f0e28fe2dcd4bd44c9e28431d0a8cec369409a"} Jul 02 23:11:10.698952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.698927 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" event={"ID":"da583992-cc89-437b-a199-67b2a476ad7a","Type":"ContainerStarted","Data":"39cf174eaadd1f56c2e06937a3a27b037474be7b25a446c6a289d51bd35df7e8"} Jul 02 23:11:10.700432 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.700414 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" event={"ID":"116d4405-27af-41ac-a5a3-b13cffd78d7b","Type":"ContainerStarted","Data":"3e5b167a595e9355baa31ef565c9a98d6e80f8c3a7d10f95d40105960f0e5e58"} Jul 02 23:11:10.701907 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.701884 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kz2k8" event={"ID":"d23b8783-dbea-4420-98a5-85a56ff2e7a9","Type":"ContainerStarted","Data":"294d5787fdd2dea717bc89d6cabe9bc6ba6ea6679b1ac5d76a20f33f63743066"} Jul 02 23:11:10.703404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.703383 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" event={"ID":"d5c7e9b5-df90-45af-bedd-66786f0fed2a","Type":"ContainerStarted","Data":"d0c639feaea548b6c30e09b8e92f74230cc8d00de1cd0230fd83d99c06b31f6d"} Jul 02 23:11:10.730881 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.730821 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" podStartSLOduration=2.284441051 podStartE2EDuration="9.730797666s" totalImagesPullingTime="7.446356615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.729931814 +0000 UTC m=+44.797544374" watchObservedRunningTime="2026-07-02 23:11:10.730797666 +0000 UTC m=+44.798410207" Jul 02 23:11:10.768741 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.768396 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kz2k8" podStartSLOduration=2.31700574 podStartE2EDuration="9.768348309s" totalImagesPullingTime="7.451342569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.749808095 +0000 UTC m=+44.817420656" watchObservedRunningTime="2026-07-02 23:11:10.768348309 +0000 UTC m=+44.835960870" Jul 02 23:11:10.787825 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.786540 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dk7wh" podStartSLOduration=7.546119314 podStartE2EDuration="9.786499252s" totalImagesPullingTime="2.240379938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.785996894 +0000 UTC m=+44.853609454" watchObservedRunningTime="2026-07-02 23:11:10.786499252 +0000 UTC m=+44.854111813" Jul 02 23:11:10.787825 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.786861 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-pdlhr" podStartSLOduration=2.30215454 podStartE2EDuration="9.786847368s" totalImagesPullingTime="7.484692828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.768048171 +0000 UTC m=+44.835660733" watchObservedRunningTime="2026-07-02 23:11:10.786847368 +0000 UTC m=+44.854459925" Jul 02 23:11:10.807907 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.807809 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" podStartSLOduration=2.359265664 podStartE2EDuration="9.807764303s" totalImagesPullingTime="7.448498639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.807156392 +0000 UTC m=+44.874768949" watchObservedRunningTime="2026-07-02 23:11:10.807764303 +0000 UTC m=+44.875376863" Jul 02 23:11:10.832527 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:10.832457 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" podStartSLOduration=3.381473553 podStartE2EDuration="10.832440311s" totalImagesPullingTime="7.450966758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:10.831137451 +0000 UTC m=+44.898750011" watchObservedRunningTime="2026-07-02 23:11:10.832440311 +0000 UTC m=+44.900052871" Jul 02 23:11:11.276926 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.276892 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7kjpw"] Jul 02 23:11:11.310485 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.310447 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7kjpw"] Jul 02 23:11:11.310689 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.310604 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.313298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.313270 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:11:11.313442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.313336 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t7j8n\"" Jul 02 23:11:11.313442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.313270 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:11:11.402670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.402635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.402834 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.402686 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-data-volume\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.402834 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.402707 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xll6\" (UniqueName: \"kubernetes.io/projected/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-api-access-5xll6\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.402834 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.402813 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.402934 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.402854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-crio-socket\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503585 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.503540 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.503615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-data-volume\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503756 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:11.503640 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:11.503756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.503651 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xll6\" (UniqueName: \"kubernetes.io/projected/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-api-access-5xll6\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503756 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:11.503708 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.003687285 +0000 UTC m=+46.071299835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : configmap "kube-rbac-proxy" not found Jul 02 23:11:11.503912 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.503760 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503912 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.503811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-crio-socket\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.503982 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:11.503941 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:11.504020 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:11.504008 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.003989606 +0000 UTC m=+46.071602155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:11.504020 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.504008 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-data-volume\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.504091 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.504033 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-crio-socket\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.515013 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.514977 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xll6\" (UniqueName: \"kubernetes.io/projected/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-api-access-5xll6\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:11.538966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.538874 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:11.538966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.538928 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:11.712021 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.711992 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:11:11.712489 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.712473 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/0.log" Jul 02 23:11:11.712619 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.712533 2558 generic.go:356] "Generic (PLEG): container finished" podID="af98f986-4915-48d1-80fa-4a8a11b99dc1" containerID="23183a32886bfeab2f94bebc2d95e7eed11cf50a85b6a8011ed02ff59340ff28" exitCode=255 Jul 02 23:11:11.712672 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.712622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" event={"ID":"af98f986-4915-48d1-80fa-4a8a11b99dc1","Type":"ContainerDied","Data":"23183a32886bfeab2f94bebc2d95e7eed11cf50a85b6a8011ed02ff59340ff28"} Jul 02 23:11:11.712720 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.712704 2558 scope.go:117] "RemoveContainer" containerID="26593a72f482d6de7be208375eabbaf439f088c6c533b02adce4c26e6f5cc008" Jul 02 23:11:11.712871 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:11.712849 2558 scope.go:117] "RemoveContainer" containerID="23183a32886bfeab2f94bebc2d95e7eed11cf50a85b6a8011ed02ff59340ff28" Jul 02 23:11:11.713078 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:11.713043 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-q62ks_openshift-console-operator(af98f986-4915-48d1-80fa-4a8a11b99dc1)\"" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" podUID="af98f986-4915-48d1-80fa-4a8a11b99dc1" Jul 02 23:11:12.009428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:12.009384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:12.009670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:12.009507 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:12.009670 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:12.009578 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:12.009670 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:12.009619 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:12.009670 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:12.009664 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:13.009643752 +0000 UTC m=+47.077256296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:12.009831 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:12.009685 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:13.009675529 +0000 UTC m=+47.077288068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : configmap "kube-rbac-proxy" not found Jul 02 23:11:12.717321 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:12.717292 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:11:12.717733 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:12.717654 2558 scope.go:117] "RemoveContainer" containerID="23183a32886bfeab2f94bebc2d95e7eed11cf50a85b6a8011ed02ff59340ff28" Jul 02 23:11:12.717838 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:12.717817 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-q62ks_openshift-console-operator(af98f986-4915-48d1-80fa-4a8a11b99dc1)\"" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" podUID="af98f986-4915-48d1-80fa-4a8a11b99dc1" Jul 02 23:11:13.018864 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:13.018832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:13.019031 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:13.018969 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:13.019077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:13.019033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:13.019077 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:13.019059 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:15.019039683 +0000 UTC m=+49.086652234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : configmap "kube-rbac-proxy" not found Jul 02 23:11:13.019145 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:13.019119 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:13.019178 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:13.019164 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:15.019152347 +0000 UTC m=+49.086764902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:13.075119 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:13.075093 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xltcn_f7603a47-9ef4-4211-adcc-1c31975fc699/dns-node-resolver/0.log" Jul 02 23:11:14.075554 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:14.075503 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tk6cr_d34d323b-43bd-4ffc-8818-e2d460f0f7be/node-ca/0.log" Jul 02 23:11:14.474629 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:14.474585 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kz2k8_d23b8783-dbea-4420-98a5-85a56ff2e7a9/serve-healthcheck-canary/0.log" Jul 02 23:11:15.038277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:15.038227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:15.038440 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:15.038363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:15.038440 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:15.038366 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:15.038440 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:15.038425 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:15.038597 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:15.038453 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:19.038436465 +0000 UTC m=+53.106049022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : configmap "kube-rbac-proxy" not found Jul 02 23:11:15.038597 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:15.038477 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:19.038464632 +0000 UTC m=+53.106077169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:17.055694 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.055652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.055728 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.055797 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.055868 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.055884 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f4567cb88-8s8bn: secret "image-registry-tls" not found Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.055875 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls podName:f3d78fe5-69c9-4854-ae87-82e7e4719dc0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.055858028 +0000 UTC m=+67.123470568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-bgp5m" (UID: "f3d78fe5-69c9-4854-ae87-82e7e4719dc0") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:17.056111 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.055933 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls podName:5b487329-8269-4bb5-a3f2-6326507f60df nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.055921013 +0000 UTC m=+67.123533549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls") pod "image-registry-f4567cb88-8s8bn" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df") : secret "image-registry-tls" not found Jul 02 23:11:17.156822 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.156779 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:17.157022 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.156933 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:17.157022 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.157004 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls podName:b406076e-94af-44ac-97c0-e2eba90effa4 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.156980729 +0000 UTC m=+67.224593269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls") pod "dns-default-rmmhq" (UID: "b406076e-94af-44ac-97c0-e2eba90effa4") : secret "dns-default-metrics-tls" not found Jul 02 23:11:17.257534 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.257490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:17.257713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.257588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:17.257713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.257627 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:17.257713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:17.257645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:17.257713 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257657 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257719 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257733 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.257712394 +0000 UTC m=+67.325324949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : secret "router-metrics-certs-default" not found Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257746 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257759 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert podName:62a2b086-a180-428e-abce-6516c0c1d001 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.257748849 +0000 UTC m=+67.325361389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-mjkj2" (UID: "62a2b086-a180-428e-abce-6516c0c1d001") : secret "networking-console-plugin-cert" not found Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257775 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle podName:f51c65fa-18ca-426b-9beb-c15a20ce99e5 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.257767409 +0000 UTC m=+67.325379949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle") pod "router-default-784c48b49-hfwdq" (UID: "f51c65fa-18ca-426b-9beb-c15a20ce99e5") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:17.257853 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:17.257789 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls podName:a9a9fb53-6114-4a00-9fef-138d8ee74f63 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.257781296 +0000 UTC m=+67.325393833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f22xn" (UID: "a9a9fb53-6114-4a00-9fef-138d8ee74f63") : secret "samples-operator-tls" not found Jul 02 23:11:19.074905 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:19.074861 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:19.075301 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:19.074990 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:19.075301 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:19.075018 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:19.075301 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:11:19.075095 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls podName:4b4aac5f-0e36-44c7-aaa5-0629e14e170c nodeName:}" failed. No retries permitted until 2026-07-02 23:11:27.075079131 +0000 UTC m=+61.142691669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7kjpw" (UID: "4b4aac5f-0e36-44c7-aaa5-0629e14e170c") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:19.075573 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:19.075555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:21.538473 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.538421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:21.538473 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.538493 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:21.539079 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.538833 2558 scope.go:117] "RemoveContainer" containerID="23183a32886bfeab2f94bebc2d95e7eed11cf50a85b6a8011ed02ff59340ff28" Jul 02 23:11:21.750077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.750051 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:11:21.750266 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.750123 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" event={"ID":"af98f986-4915-48d1-80fa-4a8a11b99dc1","Type":"ContainerStarted","Data":"b372723aeb9410e4c940380c83149a589d908acc7165ebf8c004c4e16efcd83a"} Jul 02 23:11:21.750384 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.750364 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:21.767487 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:21.767431 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" podStartSLOduration=13.323228849 podStartE2EDuration="20.767415987s" totalImagesPullingTime="7.444187138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:21.76672448 +0000 UTC m=+55.834337033" watchObservedRunningTime="2026-07-02 23:11:21.767415987 +0000 UTC m=+55.835028546" Jul 02 23:11:22.201078 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:22.201047 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-q62ks" Jul 02 23:11:26.627342 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:26.627309 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vbp4m" Jul 02 23:11:27.151578 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.151506 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:27.153846 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.153823 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b4aac5f-0e36-44c7-aaa5-0629e14e170c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7kjpw\" (UID: \"4b4aac5f-0e36-44c7-aaa5-0629e14e170c\") " pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:27.223467 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.223429 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t7j8n\"" Jul 02 23:11:27.230362 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.230337 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7kjpw" Jul 02 23:11:27.361847 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.361812 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7kjpw"] Jul 02 23:11:27.365993 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:27.365960 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4aac5f_0e36_44c7_aaa5_0629e14e170c.slice/crio-5f1eadf59f227edc9fe74752c5e4d721798191f580075874ff20b1ac9f7d227b WatchSource:0}: Error finding container 5f1eadf59f227edc9fe74752c5e4d721798191f580075874ff20b1ac9f7d227b: Status 404 returned error can't find the container with id 5f1eadf59f227edc9fe74752c5e4d721798191f580075874ff20b1ac9f7d227b Jul 02 23:11:27.770453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:27.770365 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7kjpw" event={"ID":"4b4aac5f-0e36-44c7-aaa5-0629e14e170c","Type":"ContainerStarted","Data":"5f1eadf59f227edc9fe74752c5e4d721798191f580075874ff20b1ac9f7d227b"} Jul 02 23:11:31.786688 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:31.786662 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7kjpw" event={"ID":"4b4aac5f-0e36-44c7-aaa5-0629e14e170c","Type":"ContainerStarted","Data":"e823bc692a4706ae974123c4238103e2b6a5a9e54f0ea3138fc07a241fe9404e"} Jul 02 23:11:31.786973 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:31.786720 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7kjpw" event={"ID":"4b4aac5f-0e36-44c7-aaa5-0629e14e170c","Type":"ContainerStarted","Data":"04ab71119424bcdb6b98a13247e1e7640b46cba9911af2ccf2d2c47a3743cf93"} Jul 02 23:11:32.097777 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.097677 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:11:32.098024 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.097795 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:11:32.100881 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.100861 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:11:32.101892 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.101874 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:11:32.110453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.110425 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9467f990-5431-4897-860b-e938c0872949-original-pull-secret\") pod \"global-pull-secret-syncer-2d76z\" (UID: \"9467f990-5431-4897-860b-e938c0872949\") " pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:11:32.110583 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.110462 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d89552-c124-4239-8ffc-00017b6745ca-metrics-certs\") pod \"network-metrics-daemon-ld4wq\" (UID: \"94d89552-c124-4239-8ffc-00017b6745ca\") " pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:11:32.198900 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.198859 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:32.201842 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.201811 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlm2n\" (UniqueName: \"kubernetes.io/projected/61570ae8-c673-4e96-84cc-dcec5c8c5525-kube-api-access-wlm2n\") pod \"network-check-target-628sv\" (UID: \"61570ae8-c673-4e96-84cc-dcec5c8c5525\") " pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:32.354880 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.354786 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-prtl8\"" Jul 02 23:11:32.357038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.356995 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2d76z" Jul 02 23:11:32.360529 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.360487 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4wq" Jul 02 23:11:32.365910 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.365885 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gmk5n\"" Jul 02 23:11:32.374188 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.374101 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:32.537649 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.537563 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2d76z"] Jul 02 23:11:32.541408 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:32.541382 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9467f990_5431_4897_860b_e938c0872949.slice/crio-c74bbd00f9c393091c8288b061c0045085eb1fda7dda4fe61c59d20800823292 WatchSource:0}: Error finding container c74bbd00f9c393091c8288b061c0045085eb1fda7dda4fe61c59d20800823292: Status 404 returned error can't find the container with id c74bbd00f9c393091c8288b061c0045085eb1fda7dda4fe61c59d20800823292 Jul 02 23:11:32.554459 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.554170 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-628sv"] Jul 02 23:11:32.557284 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:32.557251 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61570ae8_c673_4e96_84cc_dcec5c8c5525.slice/crio-e418543ce109924e0fd9f3d5827aa01adda6793a28ad3589098154f159f1d351 WatchSource:0}: Error finding container e418543ce109924e0fd9f3d5827aa01adda6793a28ad3589098154f159f1d351: Status 404 returned error can't find the container with id e418543ce109924e0fd9f3d5827aa01adda6793a28ad3589098154f159f1d351 Jul 02 23:11:32.759861 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.759743 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ld4wq"] Jul 02 23:11:32.762077 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:32.762044 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d89552_c124_4239_8ffc_00017b6745ca.slice/crio-04268afe4f87b1e441835aa362c6b65dc01f8f82a9bb7c33bc814baae3a08b78 WatchSource:0}: Error finding container 04268afe4f87b1e441835aa362c6b65dc01f8f82a9bb7c33bc814baae3a08b78: Status 404 returned error can't find the container with id 04268afe4f87b1e441835aa362c6b65dc01f8f82a9bb7c33bc814baae3a08b78 Jul 02 23:11:32.791685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.791647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-628sv" event={"ID":"61570ae8-c673-4e96-84cc-dcec5c8c5525","Type":"ContainerStarted","Data":"3c953700c675bee0e921f745f499e8b4f39236be490cba0631fd4e2d4eddca46"} Jul 02 23:11:32.792172 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.791710 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-628sv" event={"ID":"61570ae8-c673-4e96-84cc-dcec5c8c5525","Type":"ContainerStarted","Data":"e418543ce109924e0fd9f3d5827aa01adda6793a28ad3589098154f159f1d351"} Jul 02 23:11:32.792172 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.791779 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:11:32.793176 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.793148 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4wq" event={"ID":"94d89552-c124-4239-8ffc-00017b6745ca","Type":"ContainerStarted","Data":"04268afe4f87b1e441835aa362c6b65dc01f8f82a9bb7c33bc814baae3a08b78"} Jul 02 23:11:32.794591 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.794508 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2d76z" event={"ID":"9467f990-5431-4897-860b-e938c0872949","Type":"ContainerStarted","Data":"c74bbd00f9c393091c8288b061c0045085eb1fda7dda4fe61c59d20800823292"} Jul 02 23:11:32.806882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:32.806821 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-628sv" podStartSLOduration=66.806796447 podStartE2EDuration="1m6.806796447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:32.806492616 +0000 UTC m=+66.874105176" watchObservedRunningTime="2026-07-02 23:11:32.806796447 +0000 UTC m=+66.874409007" Jul 02 23:11:33.109432 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.109323 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:33.109432 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.109419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:33.112559 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.112506 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3d78fe5-69c9-4854-ae87-82e7e4719dc0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-bgp5m\" (UID: \"f3d78fe5-69c9-4854-ae87-82e7e4719dc0\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:33.112725 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.112596 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"image-registry-f4567cb88-8s8bn\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:33.210535 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.210480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:33.213156 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.213127 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b406076e-94af-44ac-97c0-e2eba90effa4-metrics-tls\") pod \"dns-default-rmmhq\" (UID: \"b406076e-94af-44ac-97c0-e2eba90effa4\") " pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:33.311901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.311852 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:33.312100 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.311936 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:33.312100 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.311975 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:33.312100 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.312001 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:33.312904 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.312870 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f51c65fa-18ca-426b-9beb-c15a20ce99e5-service-ca-bundle\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:33.315128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.315103 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f51c65fa-18ca-426b-9beb-c15a20ce99e5-metrics-certs\") pod \"router-default-784c48b49-hfwdq\" (UID: \"f51c65fa-18ca-426b-9beb-c15a20ce99e5\") " pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:33.315303 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.315281 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62a2b086-a180-428e-abce-6516c0c1d001-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-mjkj2\" (UID: \"62a2b086-a180-428e-abce-6516c0c1d001\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:33.315736 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.315713 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a9a9fb53-6114-4a00-9fef-138d8ee74f63-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f22xn\" (UID: \"a9a9fb53-6114-4a00-9fef-138d8ee74f63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:33.326794 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.325571 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xdfpx\"" Jul 02 23:11:33.333544 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.333312 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" Jul 02 23:11:33.363428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.363382 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jqj9h\"" Jul 02 23:11:33.372000 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.371690 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:33.418382 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.418347 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jlfnl\"" Jul 02 23:11:33.425924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.425848 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:33.464308 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.464277 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pw8md\"" Jul 02 23:11:33.472253 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.472214 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:33.491022 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.490991 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x9d77\"" Jul 02 23:11:33.498058 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.498017 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" Jul 02 23:11:33.543421 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.543301 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-l4t2n\"" Jul 02 23:11:33.549966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.549833 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" Jul 02 23:11:33.689115 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.689018 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8"] Jul 02 23:11:33.722401 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.722316 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.725826 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.725770 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8"] Jul 02 23:11:33.734574 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.734294 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 23:11:33.735643 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.735230 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 23:11:33.735643 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.735495 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 23:11:33.735850 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.735762 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 23:11:33.816442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.816392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-tmp\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.816947 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.816477 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kfsv\" (UniqueName: \"kubernetes.io/projected/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-kube-api-access-9kfsv\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.816947 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.816508 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-klusterlet-config\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.820612 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.820447 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-tx54n"] Jul 02 23:11:33.861014 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.860972 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tx54n"] Jul 02 23:11:33.861181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.861134 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:33.864978 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.864730 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:11:33.864978 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.864837 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2zt4l\"" Jul 02 23:11:33.865218 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.865157 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:11:33.918016 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.917867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgkdm\" (UniqueName: \"kubernetes.io/projected/11583f25-c965-40a0-a021-cd8a433a1834-kube-api-access-lgkdm\") pod \"downloads-685bfb5f64-tx54n\" (UID: \"11583f25-c965-40a0-a021-cd8a433a1834\") " pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:33.918016 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.917963 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-tmp\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.918866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.918036 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kfsv\" (UniqueName: \"kubernetes.io/projected/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-kube-api-access-9kfsv\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.918866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.918063 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-klusterlet-config\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.918866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.918647 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-tmp\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.921463 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.921436 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-klusterlet-config\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:33.941658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:33.941549 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kfsv\" (UniqueName: \"kubernetes.io/projected/2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03-kube-api-access-9kfsv\") pod \"klusterlet-addon-workmgr-567b88f7b9-sc7m8\" (UID: \"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:34.019292 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:34.019175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgkdm\" (UniqueName: \"kubernetes.io/projected/11583f25-c965-40a0-a021-cd8a433a1834-kube-api-access-lgkdm\") pod \"downloads-685bfb5f64-tx54n\" (UID: \"11583f25-c965-40a0-a021-cd8a433a1834\") " pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:34.031956 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:34.031738 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgkdm\" (UniqueName: \"kubernetes.io/projected/11583f25-c965-40a0-a021-cd8a433a1834-kube-api-access-lgkdm\") pod \"downloads-685bfb5f64-tx54n\" (UID: \"11583f25-c965-40a0-a021-cd8a433a1834\") " pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:34.036757 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:34.036624 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:34.173358 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:34.173322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:35.504164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:35.504036 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-784c48b49-hfwdq"] Jul 02 23:11:36.875434 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:36.875391 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf51c65fa_18ca_426b_9beb_c15a20ce99e5.slice/crio-8b576fb87466ec076b76c22f7514243a8e168229d26ebcafd7310c1e496230d6 WatchSource:0}: Error finding container 8b576fb87466ec076b76c22f7514243a8e168229d26ebcafd7310c1e496230d6: Status 404 returned error can't find the container with id 8b576fb87466ec076b76c22f7514243a8e168229d26ebcafd7310c1e496230d6 Jul 02 23:11:37.159914 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.159769 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rmmhq"] Jul 02 23:11:37.161237 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.160844 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:11:37.165181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.165144 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2"] Jul 02 23:11:37.208481 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.208443 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m"] Jul 02 23:11:37.226820 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.226787 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8"] Jul 02 23:11:37.241882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.241850 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn"] Jul 02 23:11:37.246853 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.246817 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-tx54n"] Jul 02 23:11:37.314282 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.314235 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb406076e_94af_44ac_97c0_e2eba90effa4.slice/crio-e2f216f6bfc1903641966e879f0ed9d06a812c9a4270cfad37a291649713151c WatchSource:0}: Error finding container e2f216f6bfc1903641966e879f0ed9d06a812c9a4270cfad37a291649713151c: Status 404 returned error can't find the container with id e2f216f6bfc1903641966e879f0ed9d06a812c9a4270cfad37a291649713151c Jul 02 23:11:37.315445 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.315413 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b487329_8269_4bb5_a3f2_6326507f60df.slice/crio-2efbd883006b2e521f192a0f44e02ee4be1100bddd07fdccf539ac17aa3fd4fc WatchSource:0}: Error finding container 2efbd883006b2e521f192a0f44e02ee4be1100bddd07fdccf539ac17aa3fd4fc: Status 404 returned error can't find the container with id 2efbd883006b2e521f192a0f44e02ee4be1100bddd07fdccf539ac17aa3fd4fc Jul 02 23:11:37.316344 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.316266 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a2b086_a180_428e_abce_6516c0c1d001.slice/crio-fdbfd6e0ac1c474cbd7c3a830a9e4af27ac232cde2e4b3ac4acb424a77306df0 WatchSource:0}: Error finding container fdbfd6e0ac1c474cbd7c3a830a9e4af27ac232cde2e4b3ac4acb424a77306df0: Status 404 returned error can't find the container with id fdbfd6e0ac1c474cbd7c3a830a9e4af27ac232cde2e4b3ac4acb424a77306df0 Jul 02 23:11:37.317565 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.317464 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d78fe5_69c9_4854_ae87_82e7e4719dc0.slice/crio-9eafcd7e39c5498ac0b97aec4bca7846f3d5175a306661db7ae5f759d01d9b1e WatchSource:0}: Error finding container 9eafcd7e39c5498ac0b97aec4bca7846f3d5175a306661db7ae5f759d01d9b1e: Status 404 returned error can't find the container with id 9eafcd7e39c5498ac0b97aec4bca7846f3d5175a306661db7ae5f759d01d9b1e Jul 02 23:11:37.318391 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.318354 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6ef62c_5ed7_45e5_b199_eb2fe0a08b03.slice/crio-9a7ba9d84647bce1c7bdf3e7f2511bb0faaa4a0f60968937ea7e2c46a4f00938 WatchSource:0}: Error finding container 9a7ba9d84647bce1c7bdf3e7f2511bb0faaa4a0f60968937ea7e2c46a4f00938: Status 404 returned error can't find the container with id 9a7ba9d84647bce1c7bdf3e7f2511bb0faaa4a0f60968937ea7e2c46a4f00938 Jul 02 23:11:37.320419 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:37.320287 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11583f25_c965_40a0_a021_cd8a433a1834.slice/crio-2130fbd973e21079f608847c6a50c2fee48071b4f72f83dc3662006dabe7ef03 WatchSource:0}: Error finding container 2130fbd973e21079f608847c6a50c2fee48071b4f72f83dc3662006dabe7ef03: Status 404 returned error can't find the container with id 2130fbd973e21079f608847c6a50c2fee48071b4f72f83dc3662006dabe7ef03 Jul 02 23:11:37.816978 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.816898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmmhq" event={"ID":"b406076e-94af-44ac-97c0-e2eba90effa4","Type":"ContainerStarted","Data":"e2f216f6bfc1903641966e879f0ed9d06a812c9a4270cfad37a291649713151c"} Jul 02 23:11:37.818488 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.818453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" event={"ID":"62a2b086-a180-428e-abce-6516c0c1d001","Type":"ContainerStarted","Data":"fdbfd6e0ac1c474cbd7c3a830a9e4af27ac232cde2e4b3ac4acb424a77306df0"} Jul 02 23:11:37.819997 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.819940 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tx54n" event={"ID":"11583f25-c965-40a0-a021-cd8a433a1834","Type":"ContainerStarted","Data":"2130fbd973e21079f608847c6a50c2fee48071b4f72f83dc3662006dabe7ef03"} Jul 02 23:11:37.821710 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.821685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" event={"ID":"a9a9fb53-6114-4a00-9fef-138d8ee74f63","Type":"ContainerStarted","Data":"371ccdba046ea74d4e2a8fa980f7a305ef225a97ac7b4c89074bb8ebdbce3ac7"} Jul 02 23:11:37.823049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.823000 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" event={"ID":"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03","Type":"ContainerStarted","Data":"9a7ba9d84647bce1c7bdf3e7f2511bb0faaa4a0f60968937ea7e2c46a4f00938"} Jul 02 23:11:37.825431 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.825401 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7kjpw" event={"ID":"4b4aac5f-0e36-44c7-aaa5-0629e14e170c","Type":"ContainerStarted","Data":"89fe2d89764318b5819d8a24182468bcb6fd7c8b638b3ab4c753f5e8dd877353"} Jul 02 23:11:37.827289 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.827266 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-784c48b49-hfwdq" event={"ID":"f51c65fa-18ca-426b-9beb-c15a20ce99e5","Type":"ContainerStarted","Data":"3322d6952a71b9d9d5becb6c981fa96c25ff3fbd8ca99c0444251e445853b559"} Jul 02 23:11:37.827422 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.827313 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-784c48b49-hfwdq" event={"ID":"f51c65fa-18ca-426b-9beb-c15a20ce99e5","Type":"ContainerStarted","Data":"8b576fb87466ec076b76c22f7514243a8e168229d26ebcafd7310c1e496230d6"} Jul 02 23:11:37.829613 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.829587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" event={"ID":"5b487329-8269-4bb5-a3f2-6326507f60df","Type":"ContainerStarted","Data":"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570"} Jul 02 23:11:37.829715 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.829631 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" event={"ID":"5b487329-8269-4bb5-a3f2-6326507f60df","Type":"ContainerStarted","Data":"2efbd883006b2e521f192a0f44e02ee4be1100bddd07fdccf539ac17aa3fd4fc"} Jul 02 23:11:37.829844 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.829820 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:37.831261 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.831213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" event={"ID":"f3d78fe5-69c9-4854-ae87-82e7e4719dc0","Type":"ContainerStarted","Data":"9eafcd7e39c5498ac0b97aec4bca7846f3d5175a306661db7ae5f759d01d9b1e"} Jul 02 23:11:37.834037 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.834002 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4wq" event={"ID":"94d89552-c124-4239-8ffc-00017b6745ca","Type":"ContainerStarted","Data":"aa06c8e637273dceab3f2c5b368efeb9d889686bd161c6f92acaeea1a6fcfcee"} Jul 02 23:11:37.834159 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.834061 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4wq" event={"ID":"94d89552-c124-4239-8ffc-00017b6745ca","Type":"ContainerStarted","Data":"7c4f9e0b448e223a304f3976284cb41c22b1801cd92f2e0e5f142a29b3ade8ce"} Jul 02 23:11:37.835945 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.835869 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2d76z" event={"ID":"9467f990-5431-4897-860b-e938c0872949","Type":"ContainerStarted","Data":"c6654cb85a76a232a737a2b460295f7ecef32f041a23f9d9d8e7fb26bde095d3"} Jul 02 23:11:37.848430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.848185 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7kjpw" podStartSLOduration=17.440709428 podStartE2EDuration="26.848163999s" totalImagesPullingTime="9.407454571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:11:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:37.847402408 +0000 UTC m=+71.915014967" watchObservedRunningTime="2026-07-02 23:11:37.848163999 +0000 UTC m=+71.915776556" Jul 02 23:11:37.893040 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.892206 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-784c48b49-hfwdq" podStartSLOduration=36.892183543 podStartE2EDuration="36.892183543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:37.871774008 +0000 UTC m=+71.939386568" watchObservedRunningTime="2026-07-02 23:11:37.892183543 +0000 UTC m=+71.959796103" Jul 02 23:11:37.893040 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.892710 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2d76z" podStartSLOduration=67.06673809 podStartE2EDuration="1m11.892700312s" totalImagesPullingTime="4.825962222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:37.890437944 +0000 UTC m=+71.958050504" watchObservedRunningTime="2026-07-02 23:11:37.892700312 +0000 UTC m=+71.960312866" Jul 02 23:11:37.926320 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:37.925993 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" podStartSLOduration=71.925968963 podStartE2EDuration="1m11.925968963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:37.924087892 +0000 UTC m=+71.991700452" watchObservedRunningTime="2026-07-02 23:11:37.925968963 +0000 UTC m=+71.993581523" Jul 02 23:11:38.474831 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:38.473363 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:38.478305 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:38.478061 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:38.506441 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:38.506377 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ld4wq" podStartSLOduration=68.389795183 podStartE2EDuration="1m12.506350981s" totalImagesPullingTime="4.116555798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:37.953378921 +0000 UTC m=+72.020991481" watchObservedRunningTime="2026-07-02 23:11:38.506350981 +0000 UTC m=+72.573963542" Jul 02 23:11:38.844898 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:38.844814 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:38.847707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:38.847393 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-784c48b49-hfwdq" Jul 02 23:11:44.869877 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.869798 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" event={"ID":"2a6ef62c-5ed7-45e5-b199-eb2fe0a08b03","Type":"ContainerStarted","Data":"b99c72c5ec3ef9050afeaaaf26b363c790f69994e09003e52b2951fac8e207b4"} Jul 02 23:11:44.870363 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.870086 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:44.872292 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.872267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" Jul 02 23:11:44.872425 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.872310 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" event={"ID":"f3d78fe5-69c9-4854-ae87-82e7e4719dc0","Type":"ContainerStarted","Data":"11a1a91fdb8d7c138b71f148d53c36d2c0321feac2c5c468df646e123d6b93a0"} Jul 02 23:11:44.874430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.874398 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmmhq" event={"ID":"b406076e-94af-44ac-97c0-e2eba90effa4","Type":"ContainerStarted","Data":"7882ece81dc4bdaa8bdc1562204d24c94ecccab4d8c7d986ee764ac9e062f5e5"} Jul 02 23:11:44.874600 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.874453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rmmhq" event={"ID":"b406076e-94af-44ac-97c0-e2eba90effa4","Type":"ContainerStarted","Data":"edfeddffddf51966d3002beab055e49077f2baa955a6198749f7c63a32ecdd2b"} Jul 02 23:11:44.874600 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.874502 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:44.876636 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.876605 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" event={"ID":"62a2b086-a180-428e-abce-6516c0c1d001","Type":"ContainerStarted","Data":"a5a6b62343b48886f0ce2f521e5b9b354f8f34bead01d7d40b8dff1d5446d700"} Jul 02 23:11:44.878754 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.878686 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" event={"ID":"a9a9fb53-6114-4a00-9fef-138d8ee74f63","Type":"ContainerStarted","Data":"7820b2cc78e58847aaf208d16e756cefc449d1c194548848b3baf11f7c548556"} Jul 02 23:11:44.878754 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.878729 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" event={"ID":"a9a9fb53-6114-4a00-9fef-138d8ee74f63","Type":"ContainerStarted","Data":"68c7a6871590a6c419d49bb7e60ac9cf717bc305ac25bfb6b39a1fe9ab033ea3"} Jul 02 23:11:44.890473 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.890420 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-567b88f7b9-sc7m8" podStartSLOduration=5.202659991 podStartE2EDuration="11.890402658s" totalImagesPullingTime="6.687742667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.889348626 +0000 UTC m=+78.956961186" watchObservedRunningTime="2026-07-02 23:11:44.890402658 +0000 UTC m=+78.958015220" Jul 02 23:11:44.919256 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.919201 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f22xn" podStartSLOduration=37.296167189 podStartE2EDuration="43.919184389s" totalImagesPullingTime="6.6230172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.917635795 +0000 UTC m=+78.985248354" watchObservedRunningTime="2026-07-02 23:11:44.919184389 +0000 UTC m=+78.986796949" Jul 02 23:11:44.939091 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.938989 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-bgp5m" podStartSLOduration=37.267001075 podStartE2EDuration="43.938967583s" totalImagesPullingTime="6.671966508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.937665888 +0000 UTC m=+79.005278449" watchObservedRunningTime="2026-07-02 23:11:44.938967583 +0000 UTC m=+79.006580143" Jul 02 23:11:44.963029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.962939 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rmmhq" podStartSLOduration=37.287904471 podStartE2EDuration="43.962914465s" totalImagesPullingTime="6.675009994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.960221429 +0000 UTC m=+79.027833991" watchObservedRunningTime="2026-07-02 23:11:44.962914465 +0000 UTC m=+79.030527026" Jul 02 23:11:44.998672 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:44.998585 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-mjkj2" podStartSLOduration=37.329202979 podStartE2EDuration="43.998560088s" totalImagesPullingTime="6.669357109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.996237475 +0000 UTC m=+79.063850038" watchObservedRunningTime="2026-07-02 23:11:44.998560088 +0000 UTC m=+79.066172647" Jul 02 23:11:52.322658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.322498 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9psw9"] Jul 02 23:11:52.325937 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.325910 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.335670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.335611 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:11:52.335670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.335658 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:11:52.336738 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.336601 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:11:52.337029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.337006 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-82t8x\"" Jul 02 23:11:52.337148 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.337016 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:11:52.491266 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491224 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-tls\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491266 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491534 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491318 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-sys\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491534 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-root\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491534 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491372 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-accelerators-collector-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491534 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491467 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf96m\" (UniqueName: \"kubernetes.io/projected/2d56d0c7-5897-4af0-ba44-88013b0f34c5-kube-api-access-nf96m\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491583 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-textfile\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-metrics-client-ca\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.491711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.491656 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-wtmp\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.592952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.592870 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-metrics-client-ca\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.592952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.592917 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-wtmp\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593170 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593055 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-tls\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593170 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593130 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593170 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-wtmp\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593171 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-sys\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-root\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-accelerators-collector-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf96m\" (UniqueName: \"kubernetes.io/projected/2d56d0c7-5897-4af0-ba44-88013b0f34c5-kube-api-access-nf96m\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593296 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-sys\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593579 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-textfile\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593604 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-metrics-client-ca\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593640 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-textfile\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593672 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2d56d0c7-5897-4af0-ba44-88013b0f34c5-root\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.593892 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.593866 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-accelerators-collector-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.595865 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.595839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-tls\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.595990 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.595959 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d56d0c7-5897-4af0-ba44-88013b0f34c5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.603873 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.603839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf96m\" (UniqueName: \"kubernetes.io/projected/2d56d0c7-5897-4af0-ba44-88013b0f34c5-kube-api-access-nf96m\") pod \"node-exporter-9psw9\" (UID: \"2d56d0c7-5897-4af0-ba44-88013b0f34c5\") " pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:52.637558 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:52.637504 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9psw9" Jul 02 23:11:54.885373 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:54.885332 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rmmhq" Jul 02 23:11:56.481285 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:56.481243 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:11:56.487028 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:56.486999 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:11:58.270448 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:11:58.270420 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d56d0c7_5897_4af0_ba44_88013b0f34c5.slice/crio-80aff3e7129a4eb83291b49bef74641cd0766e09081d7916f8b9659091d09dce WatchSource:0}: Error finding container 80aff3e7129a4eb83291b49bef74641cd0766e09081d7916f8b9659091d09dce: Status 404 returned error can't find the container with id 80aff3e7129a4eb83291b49bef74641cd0766e09081d7916f8b9659091d09dce Jul 02 23:11:58.943845 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:58.943748 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-tx54n" event={"ID":"11583f25-c965-40a0-a021-cd8a433a1834","Type":"ContainerStarted","Data":"87f08794eb07ce7a40eb3542690de425d37bd7d753fdec2c6aec93a6290118ea"} Jul 02 23:11:58.944032 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:58.943954 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:58.946645 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:58.946598 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9psw9" event={"ID":"2d56d0c7-5897-4af0-ba44-88013b0f34c5","Type":"ContainerStarted","Data":"80aff3e7129a4eb83291b49bef74641cd0766e09081d7916f8b9659091d09dce"} Jul 02 23:11:58.953665 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:58.953634 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-tx54n" Jul 02 23:11:58.964361 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:58.964302 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-tx54n" podStartSLOduration=4.660437508 podStartE2EDuration="25.964282836s" totalImagesPullingTime="21.303845328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:58.961695151 +0000 UTC m=+93.029307724" watchObservedRunningTime="2026-07-02 23:11:58.964282836 +0000 UTC m=+93.031895396" Jul 02 23:11:59.953395 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:59.953266 2558 generic.go:356] "Generic (PLEG): container finished" podID="2d56d0c7-5897-4af0-ba44-88013b0f34c5" containerID="f7e0dc550d1e078f4c4c7ccdafb754915af143ba865f09264aa3296b78f0d940" exitCode=0 Jul 02 23:11:59.953395 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:11:59.953381 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9psw9" event={"ID":"2d56d0c7-5897-4af0-ba44-88013b0f34c5","Type":"ContainerDied","Data":"f7e0dc550d1e078f4c4c7ccdafb754915af143ba865f09264aa3296b78f0d940"} Jul 02 23:12:00.960267 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:00.960230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9psw9" event={"ID":"2d56d0c7-5897-4af0-ba44-88013b0f34c5","Type":"ContainerStarted","Data":"6c9e83dba28b04c8d77bd90148c06925d0be1166f80312e036ef31877751b796"} Jul 02 23:12:00.960763 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:00.960287 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9psw9" event={"ID":"2d56d0c7-5897-4af0-ba44-88013b0f34c5","Type":"ContainerStarted","Data":"c5c154bfcd9d5f67e214495e0b3c46aae7f4ef9f09769affb047189370091597"} Jul 02 23:12:00.982493 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:00.982428 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9psw9" podStartSLOduration=8.030020845 podStartE2EDuration="8.982403216s" totalImagesPullingTime="863.383113ms" totalInitContainerRuntime="88.999258ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:00.980620821 +0000 UTC m=+95.048233381" watchObservedRunningTime="2026-07-02 23:12:00.982403216 +0000 UTC m=+95.050015776" Jul 02 23:12:03.802064 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:03.802027 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-628sv" Jul 02 23:12:21.504184 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.504139 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" podUID="5b487329-8269-4bb5-a3f2-6326507f60df" containerName="registry" containerID="cri-o://d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570" gracePeriod=30 Jul 02 23:12:21.796818 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.796783 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:12:21.833277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833245 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833280 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833307 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833355 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833380 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833399 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzxhs\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833441 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833485 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token\") pod \"5b487329-8269-4bb5-a3f2-6326507f60df\" (UID: \"5b487329-8269-4bb5-a3f2-6326507f60df\") " Jul 02 23:12:21.833979 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.833945 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:21.834218 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.834191 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:21.836407 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.836345 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:21.836407 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.836391 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:21.836566 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.836437 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:21.836622 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.836571 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:21.836660 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.836636 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs" (OuterVolumeSpecName: "kube-api-access-zzxhs") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "kube-api-access-zzxhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:21.842374 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.842342 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5b487329-8269-4bb5-a3f2-6326507f60df" (UID: "5b487329-8269-4bb5-a3f2-6326507f60df"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:12:21.934866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934825 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-bound-sa-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.934866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934861 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-installation-pull-secrets\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.934866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934876 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-registry-certificates\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.935121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934889 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-registry-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.935121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934900 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5b487329-8269-4bb5-a3f2-6326507f60df-trusted-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.935121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934911 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5b487329-8269-4bb5-a3f2-6326507f60df-ca-trust-extracted\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.935121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934921 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzxhs\" (UniqueName: \"kubernetes.io/projected/5b487329-8269-4bb5-a3f2-6326507f60df-kube-api-access-zzxhs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:21.935121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:21.934935 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5b487329-8269-4bb5-a3f2-6326507f60df-image-registry-private-configuration\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:12:22.042430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.042338 2558 generic.go:356] "Generic (PLEG): container finished" podID="9fb41a52-c6bd-495a-9315-1173670e2b58" containerID="1ab4391452fdadfac2ab7587f0f0e28fe2dcd4bd44c9e28431d0a8cec369409a" exitCode=0 Jul 02 23:12:22.042430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.042410 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" event={"ID":"9fb41a52-c6bd-495a-9315-1173670e2b58","Type":"ContainerDied","Data":"1ab4391452fdadfac2ab7587f0f0e28fe2dcd4bd44c9e28431d0a8cec369409a"} Jul 02 23:12:22.042864 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.042847 2558 scope.go:117] "RemoveContainer" containerID="1ab4391452fdadfac2ab7587f0f0e28fe2dcd4bd44c9e28431d0a8cec369409a" Jul 02 23:12:22.043718 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.043691 2558 generic.go:356] "Generic (PLEG): container finished" podID="5b487329-8269-4bb5-a3f2-6326507f60df" containerID="d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570" exitCode=0 Jul 02 23:12:22.043810 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.043729 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" Jul 02 23:12:22.043810 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.043756 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" event={"ID":"5b487329-8269-4bb5-a3f2-6326507f60df","Type":"ContainerDied","Data":"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570"} Jul 02 23:12:22.043810 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.043792 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4567cb88-8s8bn" event={"ID":"5b487329-8269-4bb5-a3f2-6326507f60df","Type":"ContainerDied","Data":"2efbd883006b2e521f192a0f44e02ee4be1100bddd07fdccf539ac17aa3fd4fc"} Jul 02 23:12:22.043810 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.043811 2558 scope.go:117] "RemoveContainer" containerID="d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570" Jul 02 23:12:22.056325 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.056305 2558 scope.go:117] "RemoveContainer" containerID="d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570" Jul 02 23:12:22.056663 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:12:22.056642 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570\": container with ID starting with d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570 not found: ID does not exist" containerID="d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570" Jul 02 23:12:22.056729 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.056672 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570"} err="failed to get container status \"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570\": rpc error: code = NotFound desc = could not find container \"d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570\": container with ID starting with d0c98c7df2b72b71d1434b6ade139507777e8c16cb069314270f9f4ae7a2a570 not found: ID does not exist" Jul 02 23:12:22.077652 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.077619 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:12:22.089071 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.089043 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f4567cb88-8s8bn"] Jul 02 23:12:22.444425 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:22.444348 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b487329-8269-4bb5-a3f2-6326507f60df" path="/var/lib/kubelet/pods/5b487329-8269-4bb5-a3f2-6326507f60df/volumes" Jul 02 23:12:23.049487 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:23.049449 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m5qfj" event={"ID":"9fb41a52-c6bd-495a-9315-1173670e2b58","Type":"ContainerStarted","Data":"50af74ff6f87285a9866706f7e8ae21961bfaf08862c43322d5d8656f889d50b"} Jul 02 23:12:31.079812 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:31.079768 2558 generic.go:356] "Generic (PLEG): container finished" podID="a56a6e4c-8e55-4175-8d28-b43ffe93c9b6" containerID="eb5d5c2d836780f079b01a9246bb436c40166d33afc17c08f4b62c3f3e5a6ce8" exitCode=0 Jul 02 23:12:31.080251 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:31.079837 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" event={"ID":"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6","Type":"ContainerDied","Data":"eb5d5c2d836780f079b01a9246bb436c40166d33afc17c08f4b62c3f3e5a6ce8"} Jul 02 23:12:31.080251 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:31.080141 2558 scope.go:117] "RemoveContainer" containerID="eb5d5c2d836780f079b01a9246bb436c40166d33afc17c08f4b62c3f3e5a6ce8" Jul 02 23:12:32.085640 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:32.085602 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-56ntv" event={"ID":"a56a6e4c-8e55-4175-8d28-b43ffe93c9b6","Type":"ContainerStarted","Data":"27978f3f885884bb75259353643fa4e8d3e7207c5b2aa177b8e8c3c3dd43941b"} Jul 02 23:12:36.103704 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:36.103670 2558 generic.go:356] "Generic (PLEG): container finished" podID="da583992-cc89-437b-a199-67b2a476ad7a" containerID="39cf174eaadd1f56c2e06937a3a27b037474be7b25a446c6a289d51bd35df7e8" exitCode=0 Jul 02 23:12:36.104132 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:36.103715 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" event={"ID":"da583992-cc89-437b-a199-67b2a476ad7a","Type":"ContainerDied","Data":"39cf174eaadd1f56c2e06937a3a27b037474be7b25a446c6a289d51bd35df7e8"} Jul 02 23:12:36.104132 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:36.104036 2558 scope.go:117] "RemoveContainer" containerID="39cf174eaadd1f56c2e06937a3a27b037474be7b25a446c6a289d51bd35df7e8" Jul 02 23:12:36.462618 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:36.462538 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-784c48b49-hfwdq_f51c65fa-18ca-426b-9beb-c15a20ce99e5/router/0.log" Jul 02 23:12:36.469722 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:36.469697 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kz2k8_d23b8783-dbea-4420-98a5-85a56ff2e7a9/serve-healthcheck-canary/0.log" Jul 02 23:12:37.108698 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:12:37.108662 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-w4sdf" event={"ID":"da583992-cc89-437b-a199-67b2a476ad7a","Type":"ContainerStarted","Data":"89769598b23a13749398a1c441775fd4dfb89518012d3c05a16dc6d8f3562100"} Jul 02 23:14:27.903274 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.903237 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m"] Jul 02 23:14:27.903824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.903549 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b487329-8269-4bb5-a3f2-6326507f60df" containerName="registry" Jul 02 23:14:27.903824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.903561 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b487329-8269-4bb5-a3f2-6326507f60df" containerName="registry" Jul 02 23:14:27.903824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.903625 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b487329-8269-4bb5-a3f2-6326507f60df" containerName="registry" Jul 02 23:14:27.905659 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.905641 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:27.908373 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.908338 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kp9bw\"" Jul 02 23:14:27.908603 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.908588 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:14:27.909507 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.909485 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:14:27.914161 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.914136 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:27.914263 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.914206 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9sx\" (UniqueName: \"kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:27.914263 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.914228 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:27.921657 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:27.921631 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m"] Jul 02 23:14:28.014723 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.014686 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9sx\" (UniqueName: \"kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.014895 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.014732 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.014895 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.014770 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.015180 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.015159 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.015219 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.015184 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.024931 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.024900 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9sx\" (UniqueName: \"kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.215220 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.215134 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:28.347000 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.346975 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m"] Jul 02 23:14:28.349706 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:14:28.349680 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0427a6b_c7bf_4f75_8f40_07a6d0ee7caf.slice/crio-0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8 WatchSource:0}: Error finding container 0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8: Status 404 returned error can't find the container with id 0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8 Jul 02 23:14:28.517033 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:28.516995 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerStarted","Data":"0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8"} Jul 02 23:14:35.548450 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:35.548409 2558 generic.go:356] "Generic (PLEG): container finished" podID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerID="4fa23bbf9009803cba409b07bafe0f3294623dde1ef7e6473456ac4fdd27769e" exitCode=0 Jul 02 23:14:35.549012 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:35.548451 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerDied","Data":"4fa23bbf9009803cba409b07bafe0f3294623dde1ef7e6473456ac4fdd27769e"} Jul 02 23:14:37.561019 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:37.560979 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerStarted","Data":"425d0254e503c48a96cfe023d7e492ae8e3f093452544380c01711dd3848d188"} Jul 02 23:14:38.566248 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:38.566209 2558 generic.go:356] "Generic (PLEG): container finished" podID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerID="425d0254e503c48a96cfe023d7e492ae8e3f093452544380c01711dd3848d188" exitCode=0 Jul 02 23:14:38.566655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:38.566301 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerDied","Data":"425d0254e503c48a96cfe023d7e492ae8e3f093452544380c01711dd3848d188"} Jul 02 23:14:46.599592 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:46.599549 2558 generic.go:356] "Generic (PLEG): container finished" podID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerID="cc9b02165e6bf17b072db6836ac07443f44899906ca8fdd03e6afa2d7f5cec7e" exitCode=0 Jul 02 23:14:46.599997 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:46.599603 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerDied","Data":"cc9b02165e6bf17b072db6836ac07443f44899906ca8fdd03e6afa2d7f5cec7e"} Jul 02 23:14:47.732460 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.732427 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:47.759367 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.759316 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9sx\" (UniqueName: \"kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx\") pod \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " Jul 02 23:14:47.759573 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.759552 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle\") pod \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " Jul 02 23:14:47.759646 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.759632 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util\") pod \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\" (UID: \"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf\") " Jul 02 23:14:47.760441 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.760407 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle" (OuterVolumeSpecName: "bundle") pod "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" (UID: "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:47.762221 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.762189 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx" (OuterVolumeSpecName: "kube-api-access-5j9sx") pod "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" (UID: "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf"). InnerVolumeSpecName "kube-api-access-5j9sx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:47.765113 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.765083 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util" (OuterVolumeSpecName: "util") pod "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" (UID: "d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:47.860157 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.860065 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:14:47.860157 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.860101 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9sx\" (UniqueName: \"kubernetes.io/projected/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-kube-api-access-5j9sx\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:14:47.860157 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:47.860111 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:14:48.609242 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:48.609202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" event={"ID":"d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf","Type":"ContainerDied","Data":"0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8"} Jul 02 23:14:48.609404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:48.609247 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754082c5m" Jul 02 23:14:48.609404 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:48.609254 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0938a87036545853b00fff4c64103719336b1ee7110fd0c8f6b9977fc08d1cd8" Jul 02 23:14:55.967700 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.967664 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m"] Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.967960 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="extract" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.967971 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="extract" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.967982 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="pull" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.967988 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="pull" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.968001 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="util" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.968009 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="util" Jul 02 23:14:55.968171 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.968067 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0427a6b-c7bf-4f75-8f40-07a6d0ee7caf" containerName="extract" Jul 02 23:14:55.971270 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.971249 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:55.975095 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.975071 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tffxc\"" Jul 02 23:14:55.975233 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.975075 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 23:14:55.975528 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.975496 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:14:55.989708 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:55.989674 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m"] Jul 02 23:14:56.020439 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.020395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a8cdb0f-a575-45c5-8aea-31310129b464-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.020439 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.020441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dww6b\" (UniqueName: \"kubernetes.io/projected/0a8cdb0f-a575-45c5-8aea-31310129b464-kube-api-access-dww6b\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.121640 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.121582 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a8cdb0f-a575-45c5-8aea-31310129b464-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.121784 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.121653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dww6b\" (UniqueName: \"kubernetes.io/projected/0a8cdb0f-a575-45c5-8aea-31310129b464-kube-api-access-dww6b\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.121997 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.121977 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a8cdb0f-a575-45c5-8aea-31310129b464-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.130955 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.130918 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dww6b\" (UniqueName: \"kubernetes.io/projected/0a8cdb0f-a575-45c5-8aea-31310129b464-kube-api-access-dww6b\") pod \"cert-manager-operator-controller-manager-55c444bf86-q9c5m\" (UID: \"0a8cdb0f-a575-45c5-8aea-31310129b464\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.280291 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.280250 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" Jul 02 23:14:56.422262 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.422234 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m"] Jul 02 23:14:56.425049 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:14:56.425023 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8cdb0f_a575_45c5_8aea_31310129b464.slice/crio-5cb78eacdfb8a4b205f9ae0973b92ff0e842ad8fcbbd9ef6520422adbf0885d6 WatchSource:0}: Error finding container 5cb78eacdfb8a4b205f9ae0973b92ff0e842ad8fcbbd9ef6520422adbf0885d6: Status 404 returned error can't find the container with id 5cb78eacdfb8a4b205f9ae0973b92ff0e842ad8fcbbd9ef6520422adbf0885d6 Jul 02 23:14:56.640835 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:56.640743 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" event={"ID":"0a8cdb0f-a575-45c5-8aea-31310129b464","Type":"ContainerStarted","Data":"5cb78eacdfb8a4b205f9ae0973b92ff0e842ad8fcbbd9ef6520422adbf0885d6"} Jul 02 23:14:58.652071 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:58.652024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" event={"ID":"0a8cdb0f-a575-45c5-8aea-31310129b464","Type":"ContainerStarted","Data":"c689b655e6032d1a9dc8b98ac910a7b1102a0b35dcaa1f6fe1f4aee4ed926d80"} Jul 02 23:14:58.674317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:58.674261 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-q9c5m" podStartSLOduration=2.187744612 podStartE2EDuration="3.67424528s" totalImagesPullingTime="1.486500668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:58.672443724 +0000 UTC m=+272.740056284" watchObservedRunningTime="2026-07-02 23:14:58.67424528 +0000 UTC m=+272.741857838" Jul 02 23:14:59.799986 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.799944 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tqmcr"] Jul 02 23:14:59.802276 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.802252 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.805459 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.805433 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66"] Jul 02 23:14:59.805607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.805572 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 23:14:59.805746 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.805726 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4zzqh\"" Jul 02 23:14:59.807398 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.807379 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 23:14:59.807647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.807633 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.810336 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.810310 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:14:59.810477 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.810460 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kp9bw\"" Jul 02 23:14:59.810948 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.810932 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:14:59.819301 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.819275 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tqmcr"] Jul 02 23:14:59.821924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.821899 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66"] Jul 02 23:14:59.853329 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.853288 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.853501 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.853331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vz9k\" (UniqueName: \"kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.853501 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.853454 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.853501 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.853482 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tcft\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-kube-api-access-9tcft\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.853657 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.853533 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.954224 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954187 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.954224 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954229 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tcft\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-kube-api-access-9tcft\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.954447 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.954447 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.954557 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vz9k\" (UniqueName: \"kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.954707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954686 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.954777 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.954757 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:14:59.962624 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.962594 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.962754 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.962665 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tcft\" (UniqueName: \"kubernetes.io/projected/b057353a-0100-4017-a020-aaede59525d1-kube-api-access-9tcft\") pod \"cert-manager-webhook-6486ffd997-tqmcr\" (UID: \"b057353a-0100-4017-a020-aaede59525d1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:14:59.962822 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:14:59.962803 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vz9k\" (UniqueName: \"kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:15:00.123793 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.123686 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:15:00.129916 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.129891 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:15:00.277044 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.277010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tqmcr"] Jul 02 23:15:00.279752 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:00.279724 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb057353a_0100_4017_a020_aaede59525d1.slice/crio-1eff4e73f5112b844f849e3681c807edf75212e18c41bb62d4e09ee25894066f WatchSource:0}: Error finding container 1eff4e73f5112b844f849e3681c807edf75212e18c41bb62d4e09ee25894066f: Status 404 returned error can't find the container with id 1eff4e73f5112b844f849e3681c807edf75212e18c41bb62d4e09ee25894066f Jul 02 23:15:00.296079 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.296054 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66"] Jul 02 23:15:00.298294 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:00.298262 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49925dc0_9079_4ae5_a839_bb3fc3d0b6cd.slice/crio-0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0 WatchSource:0}: Error finding container 0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0: Status 404 returned error can't find the container with id 0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0 Jul 02 23:15:00.662579 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.662488 2558 generic.go:356] "Generic (PLEG): container finished" podID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerID="033415b19a41801a06da2159e870fde390c4b714ec27e04835d17cfd5e518dd9" exitCode=0 Jul 02 23:15:00.662745 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.662576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" event={"ID":"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd","Type":"ContainerDied","Data":"033415b19a41801a06da2159e870fde390c4b714ec27e04835d17cfd5e518dd9"} Jul 02 23:15:00.662745 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.662629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" event={"ID":"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd","Type":"ContainerStarted","Data":"0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0"} Jul 02 23:15:00.663917 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:00.663888 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" event={"ID":"b057353a-0100-4017-a020-aaede59525d1","Type":"ContainerStarted","Data":"1eff4e73f5112b844f849e3681c807edf75212e18c41bb62d4e09ee25894066f"} Jul 02 23:15:04.688575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:04.688534 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" event={"ID":"b057353a-0100-4017-a020-aaede59525d1","Type":"ContainerStarted","Data":"d182c8a978e0f3c5ec917199fd293d00fc03555de6c7c3721e369dd82842e03b"} Jul 02 23:15:04.689076 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:04.688616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:15:04.692814 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:04.692786 2558 generic.go:356] "Generic (PLEG): container finished" podID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerID="7597d02c0a2805ada0ae690958a16913df616c58d5f91bdde0040589caa9ddd8" exitCode=0 Jul 02 23:15:04.692971 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:04.692836 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" event={"ID":"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd","Type":"ContainerDied","Data":"7597d02c0a2805ada0ae690958a16913df616c58d5f91bdde0040589caa9ddd8"} Jul 02 23:15:04.705722 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:04.705677 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" podStartSLOduration=2.394871858 podStartE2EDuration="5.70565639s" totalImagesPullingTime="3.310784532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:04.704932718 +0000 UTC m=+278.772545278" watchObservedRunningTime="2026-07-02 23:15:04.70565639 +0000 UTC m=+278.773268950" Jul 02 23:15:05.699698 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:05.699661 2558 generic.go:356] "Generic (PLEG): container finished" podID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerID="12c441e5efcf276ebb373910059d5d8ed4175467f132b56aef6640f86606fb08" exitCode=0 Jul 02 23:15:05.700227 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:05.699749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" event={"ID":"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd","Type":"ContainerDied","Data":"12c441e5efcf276ebb373910059d5d8ed4175467f132b56aef6640f86606fb08"} Jul 02 23:15:06.836037 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.836011 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:15:06.913648 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.913608 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle\") pod \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " Jul 02 23:15:06.913852 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.913681 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vz9k\" (UniqueName: \"kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k\") pod \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " Jul 02 23:15:06.913852 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.913726 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util\") pod \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\" (UID: \"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd\") " Jul 02 23:15:06.914143 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.914112 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle" (OuterVolumeSpecName: "bundle") pod "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" (UID: "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:15:06.915871 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.915843 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k" (OuterVolumeSpecName: "kube-api-access-8vz9k") pod "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" (UID: "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd"). InnerVolumeSpecName "kube-api-access-8vz9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:06.918293 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:06.918264 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util" (OuterVolumeSpecName: "util") pod "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" (UID: "49925dc0-9079-4ae5-a839-bb3fc3d0b6cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:15:07.015077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.015030 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:07.015077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.015064 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vz9k\" (UniqueName: \"kubernetes.io/projected/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-kube-api-access-8vz9k\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:07.015077 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.015074 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49925dc0-9079-4ae5-a839-bb3fc3d0b6cd-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:07.710378 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.710344 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" Jul 02 23:15:07.710378 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.710371 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx9r66" event={"ID":"49925dc0-9079-4ae5-a839-bb3fc3d0b6cd","Type":"ContainerDied","Data":"0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0"} Jul 02 23:15:07.710624 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:07.710421 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d3a0c3b877d5c9be3103a54ef2ccf022b2ee534bd40600be15c282e90cb63d0" Jul 02 23:15:10.237732 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.237694 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kdvfc"] Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.237976 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="util" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.237987 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="util" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.237997 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="pull" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.238002 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="pull" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.238008 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="extract" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.238014 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="extract" Jul 02 23:15:10.238127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.238061 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="49925dc0-9079-4ae5-a839-bb3fc3d0b6cd" containerName="extract" Jul 02 23:15:10.239929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.239911 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.242403 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.242373 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ng626\"" Jul 02 23:15:10.248813 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.248786 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kdvfc"] Jul 02 23:15:10.340815 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.340768 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7469\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-kube-api-access-q7469\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.341004 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.340834 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.441173 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.441126 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7469\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-kube-api-access-q7469\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.441368 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.441185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.450021 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.449979 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-bound-sa-token\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.450153 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.450099 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7469\" (UniqueName: \"kubernetes.io/projected/1f24c56f-4733-48f2-8762-5215cca9660d-kube-api-access-q7469\") pod \"cert-manager-55b4d954d5-kdvfc\" (UID: \"1f24c56f-4733-48f2-8762-5215cca9660d\") " pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.550080 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.549979 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kdvfc" Jul 02 23:15:10.683441 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.683411 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kdvfc"] Jul 02 23:15:10.685258 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:10.685227 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f24c56f_4733_48f2_8762_5215cca9660d.slice/crio-ca2de2b767e9e3366c27b48ce067ef053c857ab06bd9890970292a60af05a4cc WatchSource:0}: Error finding container ca2de2b767e9e3366c27b48ce067ef053c857ab06bd9890970292a60af05a4cc: Status 404 returned error can't find the container with id ca2de2b767e9e3366c27b48ce067ef053c857ab06bd9890970292a60af05a4cc Jul 02 23:15:10.702240 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.702215 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tqmcr" Jul 02 23:15:10.725008 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:10.724972 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kdvfc" event={"ID":"1f24c56f-4733-48f2-8762-5215cca9660d","Type":"ContainerStarted","Data":"ca2de2b767e9e3366c27b48ce067ef053c857ab06bd9890970292a60af05a4cc"} Jul 02 23:15:11.731096 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:11.731061 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kdvfc" event={"ID":"1f24c56f-4733-48f2-8762-5215cca9660d","Type":"ContainerStarted","Data":"a58b96ade510995aef3629d8a516096cb7225903fb2d4de7dbdf9bcacdf87f10"} Jul 02 23:15:11.759642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:11.759587 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kdvfc" podStartSLOduration=1.759568318 podStartE2EDuration="1.759568318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:11.758700853 +0000 UTC m=+285.826313412" watchObservedRunningTime="2026-07-02 23:15:11.759568318 +0000 UTC m=+285.827180877" Jul 02 23:15:21.036246 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.036207 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx"] Jul 02 23:15:21.039624 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.039603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.042584 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.042561 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:15:21.042694 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.042565 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:15:21.043720 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.043704 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kp9bw\"" Jul 02 23:15:21.047712 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.047377 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx"] Jul 02 23:15:21.139961 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.139916 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zcxq\" (UniqueName: \"kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.139961 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.139961 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.140173 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.139995 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.240990 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.240953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.241198 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.241032 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zcxq\" (UniqueName: \"kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.241198 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.241053 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.241427 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.241404 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.241494 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.241412 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.253017 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.252983 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zcxq\" (UniqueName: \"kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.350165 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.350060 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:21.488930 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.488875 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx"] Jul 02 23:15:21.491192 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:21.491165 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7271f02e_4f09_4404_965e_c620c8a54108.slice/crio-81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5 WatchSource:0}: Error finding container 81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5: Status 404 returned error can't find the container with id 81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5 Jul 02 23:15:21.774387 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.774354 2558 generic.go:356] "Generic (PLEG): container finished" podID="7271f02e-4f09-4404-965e-c620c8a54108" containerID="8fc5fe23b732091cda4251452d5daa4bbcf9599943450e72662fd924fd6aebe6" exitCode=0 Jul 02 23:15:21.774591 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.774427 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" event={"ID":"7271f02e-4f09-4404-965e-c620c8a54108","Type":"ContainerDied","Data":"8fc5fe23b732091cda4251452d5daa4bbcf9599943450e72662fd924fd6aebe6"} Jul 02 23:15:21.774591 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:21.774475 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" event={"ID":"7271f02e-4f09-4404-965e-c620c8a54108","Type":"ContainerStarted","Data":"81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5"} Jul 02 23:15:22.780083 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:22.780049 2558 generic.go:356] "Generic (PLEG): container finished" podID="7271f02e-4f09-4404-965e-c620c8a54108" containerID="2f95fb41a47f752feb97d45860540d5435195bf45e161792c37bb8eecd2531c6" exitCode=0 Jul 02 23:15:22.780485 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:22.780133 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" event={"ID":"7271f02e-4f09-4404-965e-c620c8a54108","Type":"ContainerDied","Data":"2f95fb41a47f752feb97d45860540d5435195bf45e161792c37bb8eecd2531c6"} Jul 02 23:15:23.787233 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:23.787198 2558 generic.go:356] "Generic (PLEG): container finished" podID="7271f02e-4f09-4404-965e-c620c8a54108" containerID="c75c26a60376aaecec6a8097168d92166951b5fa408a850f4709cd919d0f5bf7" exitCode=0 Jul 02 23:15:23.787661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:23.787286 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" event={"ID":"7271f02e-4f09-4404-965e-c620c8a54108","Type":"ContainerDied","Data":"c75c26a60376aaecec6a8097168d92166951b5fa408a850f4709cd919d0f5bf7"} Jul 02 23:15:24.924713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:24.924675 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:25.073644 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.073541 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle\") pod \"7271f02e-4f09-4404-965e-c620c8a54108\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " Jul 02 23:15:25.073644 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.073584 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util\") pod \"7271f02e-4f09-4404-965e-c620c8a54108\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " Jul 02 23:15:25.073644 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.073645 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zcxq\" (UniqueName: \"kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq\") pod \"7271f02e-4f09-4404-965e-c620c8a54108\" (UID: \"7271f02e-4f09-4404-965e-c620c8a54108\") " Jul 02 23:15:25.074484 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.074438 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle" (OuterVolumeSpecName: "bundle") pod "7271f02e-4f09-4404-965e-c620c8a54108" (UID: "7271f02e-4f09-4404-965e-c620c8a54108"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:15:25.075720 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.075694 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq" (OuterVolumeSpecName: "kube-api-access-4zcxq") pod "7271f02e-4f09-4404-965e-c620c8a54108" (UID: "7271f02e-4f09-4404-965e-c620c8a54108"). InnerVolumeSpecName "kube-api-access-4zcxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:15:25.079892 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.079856 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util" (OuterVolumeSpecName: "util") pod "7271f02e-4f09-4404-965e-c620c8a54108" (UID: "7271f02e-4f09-4404-965e-c620c8a54108"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:15:25.174817 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.174777 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:25.174817 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.174811 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7271f02e-4f09-4404-965e-c620c8a54108-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:25.174817 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.174820 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zcxq\" (UniqueName: \"kubernetes.io/projected/7271f02e-4f09-4404-965e-c620c8a54108-kube-api-access-4zcxq\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:15:25.798581 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.798534 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" event={"ID":"7271f02e-4f09-4404-965e-c620c8a54108","Type":"ContainerDied","Data":"81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5"} Jul 02 23:15:25.798758 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.798600 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81eab3d7b468088c4d83d2a3acf315f6b25aaf2cc93cba8c16cdbb6f1200b2b5" Jul 02 23:15:25.798758 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:25.798562 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlllx" Jul 02 23:15:26.336936 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:26.336907 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:15:26.339719 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:26.339684 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:15:26.349441 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:26.349409 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:15:34.868908 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.868865 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq"] Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869172 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="extract" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869184 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="extract" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869204 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="util" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869211 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="util" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869218 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="pull" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869224 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="pull" Jul 02 23:15:34.872778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.869271 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7271f02e-4f09-4404-965e-c620c8a54108" containerName="extract" Jul 02 23:15:34.873074 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.872794 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:34.891681 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.891645 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:15:34.892407 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.892386 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:15:34.898018 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.897993 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:15:34.898155 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.898032 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:15:34.898155 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.898093 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p65gt\"" Jul 02 23:15:34.928044 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:34.928001 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq"] Jul 02 23:15:35.060573 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.060501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.060765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.060635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-tmp\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.060765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.060694 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-manifests\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.060765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.060725 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.060765 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.060748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8l9w\" (UniqueName: \"kubernetes.io/projected/90c768c9-e1d6-48f3-8cbb-8b18973367dd-kube-api-access-p8l9w\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162061 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.161966 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162061 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.162013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8l9w\" (UniqueName: \"kubernetes.io/projected/90c768c9-e1d6-48f3-8cbb-8b18973367dd-kube-api-access-p8l9w\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162061 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.162053 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.162077 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-tmp\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.162102 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-manifests\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.162472 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.162456 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-manifests\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.164578 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.164547 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c768c9-e1d6-48f3-8cbb-8b18973367dd-tmp\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.164679 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.164621 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.164679 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.164657 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90c768c9-e1d6-48f3-8cbb-8b18973367dd-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.179464 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.179424 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8l9w\" (UniqueName: \"kubernetes.io/projected/90c768c9-e1d6-48f3-8cbb-8b18973367dd-kube-api-access-p8l9w\") pod \"opendatahub-operator-controller-manager-7fc9978976-4xmzq\" (UID: \"90c768c9-e1d6-48f3-8cbb-8b18973367dd\") " pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.183294 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.183268 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:35.403827 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:35.403797 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c768c9_e1d6_48f3_8cbb_8b18973367dd.slice/crio-74d6ec6e9ead8d3e0b0b836516aed87baffb1952521280eaaa6a0dfc0da0c0b0 WatchSource:0}: Error finding container 74d6ec6e9ead8d3e0b0b836516aed87baffb1952521280eaaa6a0dfc0da0c0b0: Status 404 returned error can't find the container with id 74d6ec6e9ead8d3e0b0b836516aed87baffb1952521280eaaa6a0dfc0da0c0b0 Jul 02 23:15:35.405723 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.405703 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:15:35.408657 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.408632 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq"] Jul 02 23:15:35.845987 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:35.845933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" event={"ID":"90c768c9-e1d6-48f3-8cbb-8b18973367dd","Type":"ContainerStarted","Data":"74d6ec6e9ead8d3e0b0b836516aed87baffb1952521280eaaa6a0dfc0da0c0b0"} Jul 02 23:15:38.862346 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:38.862306 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" event={"ID":"90c768c9-e1d6-48f3-8cbb-8b18973367dd","Type":"ContainerStarted","Data":"15934a4468e310a36bdbeb96161455d6aec34136893b1f53f26358ea0bb954fc"} Jul 02 23:15:39.868802 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:39.868765 2558 generic.go:356] "Generic (PLEG): container finished" podID="90c768c9-e1d6-48f3-8cbb-8b18973367dd" containerID="15934a4468e310a36bdbeb96161455d6aec34136893b1f53f26358ea0bb954fc" exitCode=0 Jul 02 23:15:39.869219 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:39.868817 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" event={"ID":"90c768c9-e1d6-48f3-8cbb-8b18973367dd","Type":"ContainerDied","Data":"15934a4468e310a36bdbeb96161455d6aec34136893b1f53f26358ea0bb954fc"} Jul 02 23:15:40.875070 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:40.875034 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" event={"ID":"90c768c9-e1d6-48f3-8cbb-8b18973367dd","Type":"ContainerStarted","Data":"3b7c2f9dce18c8d9fa6e4a5870204be3ca0139640b97d4cf75cac163ad7d6a3f"} Jul 02 23:15:40.875488 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:40.875141 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:40.942433 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:40.942375 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" podStartSLOduration=2.7813270770000003 podStartE2EDuration="6.94235797s" totalImagesPullingTime="3.151220024s" totalInitContainerRuntime="1.009810869s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:40.939988469 +0000 UTC m=+315.007601029" watchObservedRunningTime="2026-07-02 23:15:40.94235797 +0000 UTC m=+315.009970596" Jul 02 23:15:51.882173 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:51.882134 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fc9978976-4xmzq" Jul 02 23:15:56.657001 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.656966 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j866p"] Jul 02 23:15:56.660221 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.660197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.664219 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.664187 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 23:15:56.681430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.681401 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j866p"] Jul 02 23:15:56.725470 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.725434 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9zd6\" (UniqueName: \"kubernetes.io/projected/11eb27b6-abbc-4520-a929-2c29023ab3ce-kube-api-access-s9zd6\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.725670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.725487 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11eb27b6-abbc-4520-a929-2c29023ab3ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.826780 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.826735 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9zd6\" (UniqueName: \"kubernetes.io/projected/11eb27b6-abbc-4520-a929-2c29023ab3ce-kube-api-access-s9zd6\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.826955 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.826794 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11eb27b6-abbc-4520-a929-2c29023ab3ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.829207 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.829172 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11eb27b6-abbc-4520-a929-2c29023ab3ce-tls-certs\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.851834 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.851798 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9zd6\" (UniqueName: \"kubernetes.io/projected/11eb27b6-abbc-4520-a929-2c29023ab3ce-kube-api-access-s9zd6\") pod \"model-serving-api-54d89bd5fc-j866p\" (UID: \"11eb27b6-abbc-4520-a929-2c29023ab3ce\") " pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:56.907661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.907580 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg"] Jul 02 23:15:56.911094 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.911068 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:56.914538 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.914496 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 23:15:56.914658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.914553 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r6cph\"" Jul 02 23:15:56.914658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.914563 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 23:15:56.936778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.936745 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg"] Jul 02 23:15:56.970209 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:56.970173 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:57.028559 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.028500 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.028697 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.028604 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9phgb\" (UniqueName: \"kubernetes.io/projected/961e27db-bde4-4e3d-b15a-e93e125061f0-kube-api-access-9phgb\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.028756 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.028731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961e27db-bde4-4e3d-b15a-e93e125061f0-tmp\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.129191 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.129138 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9phgb\" (UniqueName: \"kubernetes.io/projected/961e27db-bde4-4e3d-b15a-e93e125061f0-kube-api-access-9phgb\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.129402 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.129208 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961e27db-bde4-4e3d-b15a-e93e125061f0-tmp\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.129402 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.129238 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.129402 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:15:57.129323 2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 02 23:15:57.129402 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:15:57.129377 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs podName:961e27db-bde4-4e3d-b15a-e93e125061f0 nodeName:}" failed. No retries permitted until 2026-07-02 23:15:57.629362052 +0000 UTC m=+331.696974589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs") pod "kube-auth-proxy-69478b7d6-xvwsg" (UID: "961e27db-bde4-4e3d-b15a-e93e125061f0") : secret "kube-auth-proxy-tls" not found Jul 02 23:15:57.132018 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.131995 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/961e27db-bde4-4e3d-b15a-e93e125061f0-tmp\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.133494 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.133467 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-j866p"] Jul 02 23:15:57.135097 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:57.135072 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11eb27b6_abbc_4520_a929_2c29023ab3ce.slice/crio-c8e1a25b269a7235c83a237bce204b1b73d06e0a2e9f7667e230c08be7afff13 WatchSource:0}: Error finding container c8e1a25b269a7235c83a237bce204b1b73d06e0a2e9f7667e230c08be7afff13: Status 404 returned error can't find the container with id c8e1a25b269a7235c83a237bce204b1b73d06e0a2e9f7667e230c08be7afff13 Jul 02 23:15:57.146953 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.146923 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9phgb\" (UniqueName: \"kubernetes.io/projected/961e27db-bde4-4e3d-b15a-e93e125061f0-kube-api-access-9phgb\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.633611 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.633570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.636063 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.636030 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/961e27db-bde4-4e3d-b15a-e93e125061f0-tls-certs\") pod \"kube-auth-proxy-69478b7d6-xvwsg\" (UID: \"961e27db-bde4-4e3d-b15a-e93e125061f0\") " pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.821595 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.821549 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" Jul 02 23:15:57.954990 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.954948 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" event={"ID":"11eb27b6-abbc-4520-a929-2c29023ab3ce","Type":"ContainerStarted","Data":"c8e1a25b269a7235c83a237bce204b1b73d06e0a2e9f7667e230c08be7afff13"} Jul 02 23:15:57.989890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:57.989859 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg"] Jul 02 23:15:57.992060 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:15:57.992028 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961e27db_bde4_4e3d_b15a_e93e125061f0.slice/crio-14589e685f80b6441b5b9abb2fefd1e3ee6332cf6c213bdc76202f448605c82b WatchSource:0}: Error finding container 14589e685f80b6441b5b9abb2fefd1e3ee6332cf6c213bdc76202f448605c82b: Status 404 returned error can't find the container with id 14589e685f80b6441b5b9abb2fefd1e3ee6332cf6c213bdc76202f448605c82b Jul 02 23:15:58.962952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:58.962911 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" event={"ID":"961e27db-bde4-4e3d-b15a-e93e125061f0","Type":"ContainerStarted","Data":"14589e685f80b6441b5b9abb2fefd1e3ee6332cf6c213bdc76202f448605c82b"} Jul 02 23:15:59.970190 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:59.970149 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" event={"ID":"11eb27b6-abbc-4520-a929-2c29023ab3ce","Type":"ContainerStarted","Data":"b74033d6d47af01ff770943e4bd7774ede54401b8165686f532f231649304336"} Jul 02 23:15:59.970692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:59.970259 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:15:59.989783 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:15:59.989725 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" podStartSLOduration=2.149900227 podStartE2EDuration="3.989708543s" totalImagesPullingTime="1.839808316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:59.988343252 +0000 UTC m=+334.055955811" watchObservedRunningTime="2026-07-02 23:15:59.989708543 +0000 UTC m=+334.057321138" Jul 02 23:16:00.979179 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:00.979141 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" event={"ID":"961e27db-bde4-4e3d-b15a-e93e125061f0","Type":"ContainerStarted","Data":"185c61e206e2970d65a2465c2c6de047d040e24d7568082be26f3693be0bffdb"} Jul 02 23:16:02.744870 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.744810 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69478b7d6-xvwsg" podStartSLOduration=4.634684416 podStartE2EDuration="6.744792137s" totalImagesPullingTime="2.110107721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:01.000593637 +0000 UTC m=+335.068206216" watchObservedRunningTime="2026-07-02 23:16:02.744792137 +0000 UTC m=+336.812404699" Jul 02 23:16:02.746336 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.746315 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c"] Jul 02 23:16:02.750034 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.750015 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.755283 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.755249 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 23:16:02.756011 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.755968 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ct8dg\"" Jul 02 23:16:02.756011 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.756002 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 23:16:02.756200 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.755983 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 23:16:02.756319 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.756299 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:16:02.756438 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.756348 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:16:02.774399 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.774368 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c"] Jul 02 23:16:02.781689 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.781655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcvsw\" (UniqueName: \"kubernetes.io/projected/c9b6c5fc-a483-4981-a4e3-165d975efc63-kube-api-access-lcvsw\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.781689 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.781695 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9b6c5fc-a483-4981-a4e3-165d975efc63-manager-config\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.781923 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.781717 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.781923 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.781835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-metrics-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.883114 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.883079 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-metrics-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.883301 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.883138 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcvsw\" (UniqueName: \"kubernetes.io/projected/c9b6c5fc-a483-4981-a4e3-165d975efc63-kube-api-access-lcvsw\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.883301 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.883250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9b6c5fc-a483-4981-a4e3-165d975efc63-manager-config\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.883369 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.883295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.883979 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.883959 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c9b6c5fc-a483-4981-a4e3-165d975efc63-manager-config\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.885699 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.885670 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-metrics-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.885809 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.885698 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9b6c5fc-a483-4981-a4e3-165d975efc63-cert\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:02.901106 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:02.901066 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcvsw\" (UniqueName: \"kubernetes.io/projected/c9b6c5fc-a483-4981-a4e3-165d975efc63-kube-api-access-lcvsw\") pod \"lws-controller-manager-9d6d7b94f-j8b6c\" (UID: \"c9b6c5fc-a483-4981-a4e3-165d975efc63\") " pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:03.060697 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:03.060592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:03.265646 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:03.265620 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c"] Jul 02 23:16:03.267342 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:16:03.267315 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b6c5fc_a483_4981_a4e3_165d975efc63.slice/crio-751e30821c12f55b7e1f90171223c6cea5ba0a047326c6df045d91fd1c7338e9 WatchSource:0}: Error finding container 751e30821c12f55b7e1f90171223c6cea5ba0a047326c6df045d91fd1c7338e9: Status 404 returned error can't find the container with id 751e30821c12f55b7e1f90171223c6cea5ba0a047326c6df045d91fd1c7338e9 Jul 02 23:16:03.993938 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:03.993897 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" event={"ID":"c9b6c5fc-a483-4981-a4e3-165d975efc63","Type":"ContainerStarted","Data":"751e30821c12f55b7e1f90171223c6cea5ba0a047326c6df045d91fd1c7338e9"} Jul 02 23:16:05.000780 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:05.000743 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" event={"ID":"c9b6c5fc-a483-4981-a4e3-165d975efc63","Type":"ContainerStarted","Data":"d10f63768818f05f44c1ead9183ecb5dd71d5eb4e40d6ef4a926855557d1a183"} Jul 02 23:16:05.001197 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:05.000830 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:16:05.047076 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:05.047006 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" podStartSLOduration=1.516099524 podStartE2EDuration="3.046989188s" totalImagesPullingTime="1.530889664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:05.044673955 +0000 UTC m=+339.112286514" watchObservedRunningTime="2026-07-02 23:16:05.046989188 +0000 UTC m=+339.114601747" Jul 02 23:16:10.985031 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:10.985000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-j866p" Jul 02 23:16:16.009870 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:16:16.009831 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9d6d7b94f-j8b6c" Jul 02 23:17:12.572912 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.572871 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7"] Jul 02 23:17:12.575638 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.575619 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.578531 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.578485 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:17:12.578671 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.578619 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rrj7j\"" Jul 02 23:17:12.579667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.579646 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:17:12.584940 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.584916 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7"] Jul 02 23:17:12.659891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.659854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.660092 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.659934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khwkr\" (UniqueName: \"kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.660092 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.659975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.761011 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.760969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.761189 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.761018 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khwkr\" (UniqueName: \"kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.761189 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.761050 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.761379 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.761357 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.761439 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.761420 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.767904 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.767876 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6"] Jul 02 23:17:12.772759 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.772729 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.774156 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.774123 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khwkr\" (UniqueName: \"kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.782168 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.782137 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6"] Jul 02 23:17:12.862270 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.862179 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.862270 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.862221 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.862270 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.862253 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26rr\" (UniqueName: \"kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.885904 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.885868 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:12.963778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.963727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.963778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.963770 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26rr\" (UniqueName: \"kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.964023 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.963909 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.964195 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.964167 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.964267 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.964210 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:12.977107 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:12.977065 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26rr\" (UniqueName: \"kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:13.023659 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.023632 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7"] Jul 02 23:17:13.025711 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:17:13.025679 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fa0b0b_0822_446c_a354_0e2b71a2154c.slice/crio-44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e WatchSource:0}: Error finding container 44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e: Status 404 returned error can't find the container with id 44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e Jul 02 23:17:13.091352 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.091323 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:13.234494 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.234467 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6"] Jul 02 23:17:13.236890 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:17:13.236864 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode116172e_3312_424d_a776_4d1cecba1e72.slice/crio-141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803 WatchSource:0}: Error finding container 141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803: Status 404 returned error can't find the container with id 141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803 Jul 02 23:17:13.315384 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.315346 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerStarted","Data":"a02b385544a339e47baaf508ebc449236d7596dd617b752365a92d5cfb5cf254"} Jul 02 23:17:13.315574 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.315399 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerStarted","Data":"141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803"} Jul 02 23:17:13.317002 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.316973 2558 generic.go:356] "Generic (PLEG): container finished" podID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerID="3767672967aad89fd36f9733bce9739134f7fd6e813bea6f10447a3d84ba593e" exitCode=0 Jul 02 23:17:13.317101 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.317052 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" event={"ID":"06fa0b0b-0822-446c-a354-0e2b71a2154c","Type":"ContainerDied","Data":"3767672967aad89fd36f9733bce9739134f7fd6e813bea6f10447a3d84ba593e"} Jul 02 23:17:13.317143 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.317102 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" event={"ID":"06fa0b0b-0822-446c-a354-0e2b71a2154c","Type":"ContainerStarted","Data":"44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e"} Jul 02 23:17:13.369106 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.369077 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n"] Jul 02 23:17:13.371814 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.371795 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.381638 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.381613 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n"] Jul 02 23:17:13.467442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.467407 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.467442 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.467444 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.467644 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.467465 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p4x5\" (UniqueName: \"kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.568660 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.568624 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.568660 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.568659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.568877 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.568770 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p4x5\" (UniqueName: \"kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.569086 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.569058 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.569086 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.569063 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.583738 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.583704 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p4x5\" (UniqueName: \"kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.697974 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.697883 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:13.837491 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.837464 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n"] Jul 02 23:17:13.839714 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:17:13.839675 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfce3e3_46a5_465f_a9a4_b3e492a0cc94.slice/crio-dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2 WatchSource:0}: Error finding container dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2: Status 404 returned error can't find the container with id dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2 Jul 02 23:17:13.962736 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.962656 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g"] Jul 02 23:17:13.965935 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.965914 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:13.974679 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:13.974644 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g"] Jul 02 23:17:14.073081 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.073056 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.073164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.073090 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqtv\" (UniqueName: \"kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.073212 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.073184 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.174052 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.174020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.174245 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.174113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.174245 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.174143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqtv\" (UniqueName: \"kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.174453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.174428 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.174608 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.174487 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.183253 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.183228 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqtv\" (UniqueName: \"kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.318637 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.318592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:14.323249 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.323220 2558 generic.go:356] "Generic (PLEG): container finished" podID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerID="c4fd79d539b602e533c51c1b800e656587aceb8e5928b34dc1f5d783f2b9fcfd" exitCode=0 Jul 02 23:17:14.323334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.323303 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" event={"ID":"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94","Type":"ContainerDied","Data":"c4fd79d539b602e533c51c1b800e656587aceb8e5928b34dc1f5d783f2b9fcfd"} Jul 02 23:17:14.323396 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.323354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" event={"ID":"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94","Type":"ContainerStarted","Data":"dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2"} Jul 02 23:17:14.324970 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.324940 2558 generic.go:356] "Generic (PLEG): container finished" podID="e116172e-3312-424d-a776-4d1cecba1e72" containerID="a02b385544a339e47baaf508ebc449236d7596dd617b752365a92d5cfb5cf254" exitCode=0 Jul 02 23:17:14.325085 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.325029 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerDied","Data":"a02b385544a339e47baaf508ebc449236d7596dd617b752365a92d5cfb5cf254"} Jul 02 23:17:14.326975 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.326951 2558 generic.go:356] "Generic (PLEG): container finished" podID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerID="9be294af7b546aa11bc7f86f184d31bedf9cc6f55f86189b6b5ea962f7e34cb4" exitCode=0 Jul 02 23:17:14.327088 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.327069 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" event={"ID":"06fa0b0b-0822-446c-a354-0e2b71a2154c","Type":"ContainerDied","Data":"9be294af7b546aa11bc7f86f184d31bedf9cc6f55f86189b6b5ea962f7e34cb4"} Jul 02 23:17:14.474816 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:14.474697 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g"] Jul 02 23:17:14.476961 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:17:14.476928 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ae7ee9_1667_4dee_b7a8_e9169daa4928.slice/crio-29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c WatchSource:0}: Error finding container 29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c: Status 404 returned error can't find the container with id 29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c Jul 02 23:17:15.333885 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.333793 2558 generic.go:356] "Generic (PLEG): container finished" podID="e116172e-3312-424d-a776-4d1cecba1e72" containerID="177e823fda4bb0301f9b4ac3e14c981c2995288df0df3224640e3c3e1a6ac7ea" exitCode=0 Jul 02 23:17:15.333885 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.333868 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerDied","Data":"177e823fda4bb0301f9b4ac3e14c981c2995288df0df3224640e3c3e1a6ac7ea"} Jul 02 23:17:15.336093 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.336068 2558 generic.go:356] "Generic (PLEG): container finished" podID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerID="e08e21014221feb6cc27bd59d0293159a2d597a21382fe3ce314f3586295aa92" exitCode=0 Jul 02 23:17:15.336189 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.336154 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" event={"ID":"06fa0b0b-0822-446c-a354-0e2b71a2154c","Type":"ContainerDied","Data":"e08e21014221feb6cc27bd59d0293159a2d597a21382fe3ce314f3586295aa92"} Jul 02 23:17:15.337572 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.337536 2558 generic.go:356] "Generic (PLEG): container finished" podID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerID="fd77abfbb51eeaa10a3bacc3d2b527f5194709cebcd047a66bfa0f4f47001a3f" exitCode=0 Jul 02 23:17:15.337665 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.337614 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" event={"ID":"04ae7ee9-1667-4dee-b7a8-e9169daa4928","Type":"ContainerDied","Data":"fd77abfbb51eeaa10a3bacc3d2b527f5194709cebcd047a66bfa0f4f47001a3f"} Jul 02 23:17:15.337665 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.337652 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" event={"ID":"04ae7ee9-1667-4dee-b7a8-e9169daa4928","Type":"ContainerStarted","Data":"29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c"} Jul 02 23:17:15.339672 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.339644 2558 generic.go:356] "Generic (PLEG): container finished" podID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerID="9787528522c3e0e40c0e88e343bc547abfbcb4d05e277a89b6dc92aa3f6bdc07" exitCode=0 Jul 02 23:17:15.339771 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:15.339699 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" event={"ID":"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94","Type":"ContainerDied","Data":"9787528522c3e0e40c0e88e343bc547abfbcb4d05e277a89b6dc92aa3f6bdc07"} Jul 02 23:17:16.347371 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.347333 2558 generic.go:356] "Generic (PLEG): container finished" podID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerID="ea1eaeb15a4cc094ac7fc614b5b4b1b4c1bbc6e4020b269190048f6a63cc2cc8" exitCode=0 Jul 02 23:17:16.347837 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.347422 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" event={"ID":"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94","Type":"ContainerDied","Data":"ea1eaeb15a4cc094ac7fc614b5b4b1b4c1bbc6e4020b269190048f6a63cc2cc8"} Jul 02 23:17:16.349668 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.349642 2558 generic.go:356] "Generic (PLEG): container finished" podID="e116172e-3312-424d-a776-4d1cecba1e72" containerID="c05588c294fea57b83f47406cb429550413f89bc9b37989c39abbdee0b3e7468" exitCode=0 Jul 02 23:17:16.349793 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.349725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerDied","Data":"c05588c294fea57b83f47406cb429550413f89bc9b37989c39abbdee0b3e7468"} Jul 02 23:17:16.351465 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.351417 2558 generic.go:356] "Generic (PLEG): container finished" podID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerID="64b846df018d36bc9b018690b18d05975a1539ccfc62d8379968aa03750d53c8" exitCode=0 Jul 02 23:17:16.351465 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.351449 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" event={"ID":"04ae7ee9-1667-4dee-b7a8-e9169daa4928","Type":"ContainerDied","Data":"64b846df018d36bc9b018690b18d05975a1539ccfc62d8379968aa03750d53c8"} Jul 02 23:17:16.484578 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.484552 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:16.594268 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.594231 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle\") pod \"06fa0b0b-0822-446c-a354-0e2b71a2154c\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " Jul 02 23:17:16.594429 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.594287 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util\") pod \"06fa0b0b-0822-446c-a354-0e2b71a2154c\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " Jul 02 23:17:16.594429 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.594313 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khwkr\" (UniqueName: \"kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr\") pod \"06fa0b0b-0822-446c-a354-0e2b71a2154c\" (UID: \"06fa0b0b-0822-446c-a354-0e2b71a2154c\") " Jul 02 23:17:16.594945 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.594918 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle" (OuterVolumeSpecName: "bundle") pod "06fa0b0b-0822-446c-a354-0e2b71a2154c" (UID: "06fa0b0b-0822-446c-a354-0e2b71a2154c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:16.596551 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.596524 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr" (OuterVolumeSpecName: "kube-api-access-khwkr") pod "06fa0b0b-0822-446c-a354-0e2b71a2154c" (UID: "06fa0b0b-0822-446c-a354-0e2b71a2154c"). InnerVolumeSpecName "kube-api-access-khwkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:17:16.599398 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.599352 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util" (OuterVolumeSpecName: "util") pod "06fa0b0b-0822-446c-a354-0e2b71a2154c" (UID: "06fa0b0b-0822-446c-a354-0e2b71a2154c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:16.695416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.695364 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:16.695416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.695411 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06fa0b0b-0822-446c-a354-0e2b71a2154c-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:16.695416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:16.695422 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khwkr\" (UniqueName: \"kubernetes.io/projected/06fa0b0b-0822-446c-a354-0e2b71a2154c-kube-api-access-khwkr\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.358479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.358448 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" Jul 02 23:17:17.358918 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.358478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7" event={"ID":"06fa0b0b-0822-446c-a354-0e2b71a2154c","Type":"ContainerDied","Data":"44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e"} Jul 02 23:17:17.358918 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.358549 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44f15e144f3ff5064cb5dc4cd8f95d9efaae101e9aa89c0e0dfeffc40e9cd36e" Jul 02 23:17:17.360670 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.360645 2558 generic.go:356] "Generic (PLEG): container finished" podID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerID="c72eae6b7f480c7e41a71e6b9b0270ed5e078311a944b58d429c32926b4f0d6d" exitCode=0 Jul 02 23:17:17.360788 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.360734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" event={"ID":"04ae7ee9-1667-4dee-b7a8-e9169daa4928","Type":"ContainerDied","Data":"c72eae6b7f480c7e41a71e6b9b0270ed5e078311a944b58d429c32926b4f0d6d"} Jul 02 23:17:17.517062 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.517032 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:17.519222 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.519196 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:17.603586 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603545 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle\") pod \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " Jul 02 23:17:17.603586 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603591 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p4x5\" (UniqueName: \"kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5\") pod \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " Jul 02 23:17:17.603793 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603765 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle\") pod \"e116172e-3312-424d-a776-4d1cecba1e72\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " Jul 02 23:17:17.603826 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603815 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util\") pod \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\" (UID: \"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94\") " Jul 02 23:17:17.603863 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603853 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m26rr\" (UniqueName: \"kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr\") pod \"e116172e-3312-424d-a776-4d1cecba1e72\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " Jul 02 23:17:17.603900 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.603873 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util\") pod \"e116172e-3312-424d-a776-4d1cecba1e72\" (UID: \"e116172e-3312-424d-a776-4d1cecba1e72\") " Jul 02 23:17:17.604352 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.604317 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle" (OuterVolumeSpecName: "bundle") pod "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" (UID: "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:17.604465 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.604347 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle" (OuterVolumeSpecName: "bundle") pod "e116172e-3312-424d-a776-4d1cecba1e72" (UID: "e116172e-3312-424d-a776-4d1cecba1e72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:17.605879 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.605857 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5" (OuterVolumeSpecName: "kube-api-access-4p4x5") pod "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" (UID: "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94"). InnerVolumeSpecName "kube-api-access-4p4x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:17:17.606058 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.606034 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr" (OuterVolumeSpecName: "kube-api-access-m26rr") pod "e116172e-3312-424d-a776-4d1cecba1e72" (UID: "e116172e-3312-424d-a776-4d1cecba1e72"). InnerVolumeSpecName "kube-api-access-m26rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:17:17.609589 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.609535 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util" (OuterVolumeSpecName: "util") pod "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" (UID: "8cfce3e3-46a5-465f-a9a4-b3e492a0cc94"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:17.609683 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.609568 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util" (OuterVolumeSpecName: "util") pod "e116172e-3312-424d-a776-4d1cecba1e72" (UID: "e116172e-3312-424d-a776-4d1cecba1e72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:17.705388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705349 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.705388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705382 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m26rr\" (UniqueName: \"kubernetes.io/projected/e116172e-3312-424d-a776-4d1cecba1e72-kube-api-access-m26rr\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.705388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705393 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.705388 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705401 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.705706 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705411 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p4x5\" (UniqueName: \"kubernetes.io/projected/8cfce3e3-46a5-465f-a9a4-b3e492a0cc94-kube-api-access-4p4x5\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:17.705706 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:17.705420 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e116172e-3312-424d-a776-4d1cecba1e72-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:18.367045 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.367009 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" Jul 02 23:17:18.367045 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.367022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n" event={"ID":"8cfce3e3-46a5-465f-a9a4-b3e492a0cc94","Type":"ContainerDied","Data":"dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2"} Jul 02 23:17:18.367536 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.367079 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfa3219bf432427bfc2a4c471dccfdecc2827425c0fe6e97eab19ddc9710e5e2" Jul 02 23:17:18.369032 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.369005 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" event={"ID":"e116172e-3312-424d-a776-4d1cecba1e72","Type":"ContainerDied","Data":"141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803"} Jul 02 23:17:18.369032 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.369031 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6" Jul 02 23:17:18.369234 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.369045 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="141d9bd5f751ca4411330890cb3adaaa4d9cdcb4825a821024bbd75ea4d8f803" Jul 02 23:17:18.504193 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.504160 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:17:18.613989 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.613947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle\") pod \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " Jul 02 23:17:18.614160 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.614007 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util\") pod \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " Jul 02 23:17:18.614160 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.614034 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftqtv\" (UniqueName: \"kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv\") pod \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\" (UID: \"04ae7ee9-1667-4dee-b7a8-e9169daa4928\") " Jul 02 23:17:18.614595 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.614564 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle" (OuterVolumeSpecName: "bundle") pod "04ae7ee9-1667-4dee-b7a8-e9169daa4928" (UID: "04ae7ee9-1667-4dee-b7a8-e9169daa4928"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:18.616121 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.616101 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv" (OuterVolumeSpecName: "kube-api-access-ftqtv") pod "04ae7ee9-1667-4dee-b7a8-e9169daa4928" (UID: "04ae7ee9-1667-4dee-b7a8-e9169daa4928"). InnerVolumeSpecName "kube-api-access-ftqtv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:17:18.621845 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.621779 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util" (OuterVolumeSpecName: "util") pod "04ae7ee9-1667-4dee-b7a8-e9169daa4928" (UID: "04ae7ee9-1667-4dee-b7a8-e9169daa4928"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:17:18.714890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.714847 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:18.714890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.714883 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04ae7ee9-1667-4dee-b7a8-e9169daa4928-util\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:18.714890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:18.714893 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftqtv\" (UniqueName: \"kubernetes.io/projected/04ae7ee9-1667-4dee-b7a8-e9169daa4928-kube-api-access-ftqtv\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:17:19.376235 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:19.376192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" event={"ID":"04ae7ee9-1667-4dee-b7a8-e9169daa4928","Type":"ContainerDied","Data":"29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c"} Jul 02 23:17:19.376717 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:19.376252 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29c47db78f64f0bcc86ab2be24e7c319925149f50be6bb1ea412b8b33020e17c" Jul 02 23:17:19.376717 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:17:19.376210 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g" Jul 02 23:18:29.482012 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.481972 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482313 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482327 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482341 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482346 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482354 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482359 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482367 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482372 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482383 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482391 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482402 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482407 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482414 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482419 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482424 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482429 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="util" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482435 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482439 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="extract" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482445 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482450 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482458 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="pull" Jul 02 23:18:29.482453 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482466 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="pull" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482475 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="extract" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482481 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="extract" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482559 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfce3e3-46a5-465f-a9a4-b3e492a0cc94" containerName="extract" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482569 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06fa0b0b-0822-446c-a354-0e2b71a2154c" containerName="extract" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482574 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e116172e-3312-424d-a776-4d1cecba1e72" containerName="extract" Jul 02 23:18:29.483128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.482579 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ae7ee9-1667-4dee-b7a8-e9169daa4928" containerName="extract" Jul 02 23:18:29.485709 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.485688 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.489156 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.489116 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 23:18:29.489711 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.489690 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:18:29.490403 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.490382 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rrj7j\"" Jul 02 23:18:29.491928 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.491902 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:18:29.500040 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.500010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:18:29.535215 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.535180 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:18:29.619392 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.619357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/43655c91-5405-4775-9666-e909ba94e8f7-config-file\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.619587 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.619457 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h76nd\" (UniqueName: \"kubernetes.io/projected/43655c91-5405-4775-9666-e909ba94e8f7-kube-api-access-h76nd\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.719914 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.719879 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/43655c91-5405-4775-9666-e909ba94e8f7-config-file\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.720112 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.719951 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h76nd\" (UniqueName: \"kubernetes.io/projected/43655c91-5405-4775-9666-e909ba94e8f7-kube-api-access-h76nd\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.720591 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.720570 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/43655c91-5405-4775-9666-e909ba94e8f7-config-file\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.728592 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.728557 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h76nd\" (UniqueName: \"kubernetes.io/projected/43655c91-5405-4775-9666-e909ba94e8f7-kube-api-access-h76nd\") pod \"limitador-limitador-54d6d988d5-g4htq\" (UID: \"43655c91-5405-4775-9666-e909ba94e8f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.795810 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.795712 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:29.810907 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.810876 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:29.815239 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.815211 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:29.817781 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.817756 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2lq5g\"" Jul 02 23:18:29.820068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.820040 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:29.921034 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.920995 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69hws\" (UniqueName: \"kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws\") pod \"authorino-58cd955675-rl4kk\" (UID: \"6360995e-c53d-485c-882c-ab7d25dbc741\") " pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:29.946788 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:29.946756 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:18:29.947912 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:18:29.947885 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43655c91_5405_4775_9666_e909ba94e8f7.slice/crio-64e4cc85ffd8906042f0e916b5ea22b7dfc5861e48460ccbd569c913d74844e0 WatchSource:0}: Error finding container 64e4cc85ffd8906042f0e916b5ea22b7dfc5861e48460ccbd569c913d74844e0: Status 404 returned error can't find the container with id 64e4cc85ffd8906042f0e916b5ea22b7dfc5861e48460ccbd569c913d74844e0 Jul 02 23:18:30.022427 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.022385 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69hws\" (UniqueName: \"kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws\") pod \"authorino-58cd955675-rl4kk\" (UID: \"6360995e-c53d-485c-882c-ab7d25dbc741\") " pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:30.032274 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.032241 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69hws\" (UniqueName: \"kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws\") pod \"authorino-58cd955675-rl4kk\" (UID: \"6360995e-c53d-485c-882c-ab7d25dbc741\") " pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:30.130933 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.130838 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:30.266446 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.266423 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:30.267963 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:18:30.267940 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6360995e_c53d_485c_882c_ab7d25dbc741.slice/crio-5554e4a8d65c5450cdd9ea5bf51d469925079bffa6ff7c83bba784272076390a WatchSource:0}: Error finding container 5554e4a8d65c5450cdd9ea5bf51d469925079bffa6ff7c83bba784272076390a: Status 404 returned error can't find the container with id 5554e4a8d65c5450cdd9ea5bf51d469925079bffa6ff7c83bba784272076390a Jul 02 23:18:30.730065 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.730021 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rl4kk" event={"ID":"6360995e-c53d-485c-882c-ab7d25dbc741","Type":"ContainerStarted","Data":"5554e4a8d65c5450cdd9ea5bf51d469925079bffa6ff7c83bba784272076390a"} Jul 02 23:18:30.731535 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:30.731487 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" event={"ID":"43655c91-5405-4775-9666-e909ba94e8f7","Type":"ContainerStarted","Data":"64e4cc85ffd8906042f0e916b5ea22b7dfc5861e48460ccbd569c913d74844e0"} Jul 02 23:18:34.094278 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.094233 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:34.756905 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.756863 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rl4kk" event={"ID":"6360995e-c53d-485c-882c-ab7d25dbc741","Type":"ContainerStarted","Data":"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b"} Jul 02 23:18:34.757119 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.756870 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rl4kk" podUID="6360995e-c53d-485c-882c-ab7d25dbc741" containerName="authorino" containerID="cri-o://c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b" gracePeriod=30 Jul 02 23:18:34.758694 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.758666 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" event={"ID":"43655c91-5405-4775-9666-e909ba94e8f7","Type":"ContainerStarted","Data":"8baf806d05a3102e87651addaaef88c250c0647531eae5e15d8672e0d5e35f4c"} Jul 02 23:18:34.758812 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.758749 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:18:34.771708 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.771666 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rl4kk" podStartSLOduration=1.791611789 podStartE2EDuration="5.771642631s" totalImagesPullingTime="3.980030842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:34.770870788 +0000 UTC m=+488.838483358" watchObservedRunningTime="2026-07-02 23:18:34.771642631 +0000 UTC m=+488.839255190" Jul 02 23:18:34.786921 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:34.786868 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" podStartSLOduration=1.429124007 podStartE2EDuration="5.786851019s" totalImagesPullingTime="4.357727012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:34.786397597 +0000 UTC m=+488.854010221" watchObservedRunningTime="2026-07-02 23:18:34.786851019 +0000 UTC m=+488.854463608" Jul 02 23:18:35.000195 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.000166 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:35.171787 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.171701 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69hws\" (UniqueName: \"kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws\") pod \"6360995e-c53d-485c-882c-ab7d25dbc741\" (UID: \"6360995e-c53d-485c-882c-ab7d25dbc741\") " Jul 02 23:18:35.173778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.173751 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws" (OuterVolumeSpecName: "kube-api-access-69hws") pod "6360995e-c53d-485c-882c-ab7d25dbc741" (UID: "6360995e-c53d-485c-882c-ab7d25dbc741"). InnerVolumeSpecName "kube-api-access-69hws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:18:35.272683 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.272651 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69hws\" (UniqueName: \"kubernetes.io/projected/6360995e-c53d-485c-882c-ab7d25dbc741-kube-api-access-69hws\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:18:35.764867 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.764830 2558 generic.go:356] "Generic (PLEG): container finished" podID="6360995e-c53d-485c-882c-ab7d25dbc741" containerID="c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b" exitCode=0 Jul 02 23:18:35.765049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.764872 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rl4kk" Jul 02 23:18:35.765049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.764920 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rl4kk" event={"ID":"6360995e-c53d-485c-882c-ab7d25dbc741","Type":"ContainerDied","Data":"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b"} Jul 02 23:18:35.765049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.764962 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rl4kk" event={"ID":"6360995e-c53d-485c-882c-ab7d25dbc741","Type":"ContainerDied","Data":"5554e4a8d65c5450cdd9ea5bf51d469925079bffa6ff7c83bba784272076390a"} Jul 02 23:18:35.765049 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.764983 2558 scope.go:117] "RemoveContainer" containerID="c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b" Jul 02 23:18:35.774053 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.774032 2558 scope.go:117] "RemoveContainer" containerID="c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b" Jul 02 23:18:35.774371 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:18:35.774352 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b\": container with ID starting with c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b not found: ID does not exist" containerID="c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b" Jul 02 23:18:35.774419 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.774396 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b"} err="failed to get container status \"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b\": rpc error: code = NotFound desc = could not find container \"c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b\": container with ID starting with c3c67c8a4d5e82e01ed8dff5b0db2fe65a9ca0b83cce011aade3b7c64266d53b not found: ID does not exist" Jul 02 23:18:35.786835 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.786803 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:35.792760 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:35.792725 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rl4kk"] Jul 02 23:18:36.445118 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:36.445081 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6360995e-c53d-485c-882c-ab7d25dbc741" path="/var/lib/kubelet/pods/6360995e-c53d-485c-882c-ab7d25dbc741/volumes" Jul 02 23:18:45.767044 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:18:45.767013 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-g4htq" Jul 02 23:19:45.143936 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:19:45.143901 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:20:05.467398 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:05.467360 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:20:17.766961 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:17.766923 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:20:25.563349 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:25.563309 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:20:26.374585 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:26.374543 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:20:26.374787 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:26.374678 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:20:33.371879 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.371845 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx"] Jul 02 23:20:33.372304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.372201 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6360995e-c53d-485c-882c-ab7d25dbc741" containerName="authorino" Jul 02 23:20:33.372304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.372215 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6360995e-c53d-485c-882c-ab7d25dbc741" containerName="authorino" Jul 02 23:20:33.372304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.372286 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6360995e-c53d-485c-882c-ab7d25dbc741" containerName="authorino" Jul 02 23:20:33.375262 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.375244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376574 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376546 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376696 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376696 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376610 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376696 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376696 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376672 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.376864 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.376694 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwhwh\" (UniqueName: \"kubernetes.io/projected/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-kube-api-access-fwhwh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.379488 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.379466 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 23:20:33.379655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.379533 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:20:33.379655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.379578 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8c2j9\"" Jul 02 23:20:33.379655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.379579 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:20:33.384275 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.384254 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx"] Jul 02 23:20:33.477882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.477882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477890 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477962 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478130 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.477987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwhwh\" (UniqueName: \"kubernetes.io/projected/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-kube-api-access-fwhwh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478403 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.478376 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478403 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.478402 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.478575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.478425 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.480216 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.480187 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.480412 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.480397 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.485925 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.485899 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwhwh\" (UniqueName: \"kubernetes.io/projected/e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850-kube-api-access-fwhwh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx\" (UID: \"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.686202 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.686112 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:33.826110 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:33.826082 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx"] Jul 02 23:20:33.827825 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:20:33.827797 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a5f2cf_ed6e_4f27_9bad_2ac4100ee850.slice/crio-e83ddae50411fafdefea2db4a320530838ba43202d73b9598155358191526aaa WatchSource:0}: Error finding container e83ddae50411fafdefea2db4a320530838ba43202d73b9598155358191526aaa: Status 404 returned error can't find the container with id e83ddae50411fafdefea2db4a320530838ba43202d73b9598155358191526aaa Jul 02 23:20:34.366822 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:34.366747 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" event={"ID":"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850","Type":"ContainerStarted","Data":"e83ddae50411fafdefea2db4a320530838ba43202d73b9598155358191526aaa"} Jul 02 23:20:35.164232 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:35.164141 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:20:35.373169 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:35.373113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" event={"ID":"e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850","Type":"ContainerStarted","Data":"df840dd717060b60cee7e62cec6e33a6017555437725be76affde9e3f58ef3d5"} Jul 02 23:20:35.392397 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:35.392339 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" podStartSLOduration=1.399461495 podStartE2EDuration="2.392323422s" totalImagesPullingTime="992.861927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:20:35.390825413 +0000 UTC m=+609.458437974" watchObservedRunningTime="2026-07-02 23:20:35.392323422 +0000 UTC m=+609.459935980" Jul 02 23:20:43.686717 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:43.686667 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:43.687127 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:43.686752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:43.699298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:43.699271 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:20:44.431578 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:20:44.431540 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx" Jul 02 23:21:01.854244 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:01.854154 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:21:06.862674 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.862641 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx"] Jul 02 23:21:06.868841 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.868818 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.872917 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.872885 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 23:21:06.875682 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.875658 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx"] Jul 02 23:21:06.950329 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950286 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.950329 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950328 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.950667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950394 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.950667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a5a43c-b566-4b4f-a09d-f015121e8631-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.950667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950463 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:06.950667 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:06.950480 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9xtb\" (UniqueName: \"kubernetes.io/projected/d6a5a43c-b566-4b4f-a09d-f015121e8631-kube-api-access-d9xtb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.050890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.050843 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.050890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.050897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.050934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.050953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a5a43c-b566-4b4f-a09d-f015121e8631-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.050992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.051008 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9xtb\" (UniqueName: \"kubernetes.io/projected/d6a5a43c-b566-4b4f-a09d-f015121e8631-kube-api-access-d9xtb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051330 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.051277 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051389 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.051369 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.051490 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.051468 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.053187 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.053161 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6a5a43c-b566-4b4f-a09d-f015121e8631-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.053366 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.053348 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6a5a43c-b566-4b4f-a09d-f015121e8631-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.066479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.066452 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9xtb\" (UniqueName: \"kubernetes.io/projected/d6a5a43c-b566-4b4f-a09d-f015121e8631-kube-api-access-d9xtb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx\" (UID: \"d6a5a43c-b566-4b4f-a09d-f015121e8631\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.179330 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.179239 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:07.322075 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.322043 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx"] Jul 02 23:21:07.323908 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:21:07.323880 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6a5a43c_b566_4b4f_a09d_f015121e8631.slice/crio-89ece95aa6caadedafcda93bb0f1b3e37373cfcf09d5fb2ced5550155084e764 WatchSource:0}: Error finding container 89ece95aa6caadedafcda93bb0f1b3e37373cfcf09d5fb2ced5550155084e764: Status 404 returned error can't find the container with id 89ece95aa6caadedafcda93bb0f1b3e37373cfcf09d5fb2ced5550155084e764 Jul 02 23:21:07.325182 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.325163 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:21:07.534794 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:07.534767 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" event={"ID":"d6a5a43c-b566-4b4f-a09d-f015121e8631","Type":"ContainerStarted","Data":"89ece95aa6caadedafcda93bb0f1b3e37373cfcf09d5fb2ced5550155084e764"} Jul 02 23:21:08.541523 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:08.541488 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" event={"ID":"d6a5a43c-b566-4b4f-a09d-f015121e8631","Type":"ContainerStarted","Data":"9c9d35b8af85093d78ae4c6837e44be4ff2befca15910ff911e16110f8e72739"} Jul 02 23:21:08.559401 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:08.559346 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" podStartSLOduration=2.372055866 podStartE2EDuration="2.559331273s" totalImagesPullingTime="187.275407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:21:08.557438256 +0000 UTC m=+642.625050815" watchObservedRunningTime="2026-07-02 23:21:08.559331273 +0000 UTC m=+642.626943823" Jul 02 23:21:10.758854 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:10.758815 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:21:17.179752 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:17.179703 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:17.180141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:17.179772 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:17.192317 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:17.192285 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:21:17.597871 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:21:17.597839 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx" Jul 02 23:22:29.074416 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.074336 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5db7bc6d5f-r857s"] Jul 02 23:22:29.077477 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.077459 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.081279 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.081250 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2lq5g\"" Jul 02 23:22:29.081411 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.081302 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 23:22:29.085356 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.085334 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5db7bc6d5f-r857s"] Jul 02 23:22:29.195803 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.195767 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl6mw\" (UniqueName: \"kubernetes.io/projected/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-kube-api-access-kl6mw\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.196010 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.195872 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-tls-cert\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.296989 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.296950 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl6mw\" (UniqueName: \"kubernetes.io/projected/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-kube-api-access-kl6mw\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.297149 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.297041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-tls-cert\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.299421 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.299396 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-tls-cert\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.305399 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.305365 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl6mw\" (UniqueName: \"kubernetes.io/projected/f2f4c5e2-71fc-4186-a190-2b77cf025bd9-kube-api-access-kl6mw\") pod \"authorino-5db7bc6d5f-r857s\" (UID: \"f2f4c5e2-71fc-4186-a190-2b77cf025bd9\") " pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.387269 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.387175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5db7bc6d5f-r857s" Jul 02 23:22:29.522510 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.522482 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5db7bc6d5f-r857s"] Jul 02 23:22:29.524233 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:22:29.524194 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2f4c5e2_71fc_4186_a190_2b77cf025bd9.slice/crio-0d3667904ab51f281c8cdf2276dbbcc23673273f85df29e0281eab8105ccf86e WatchSource:0}: Error finding container 0d3667904ab51f281c8cdf2276dbbcc23673273f85df29e0281eab8105ccf86e: Status 404 returned error can't find the container with id 0d3667904ab51f281c8cdf2276dbbcc23673273f85df29e0281eab8105ccf86e Jul 02 23:22:29.958942 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.958902 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db7bc6d5f-r857s" event={"ID":"f2f4c5e2-71fc-4186-a190-2b77cf025bd9","Type":"ContainerStarted","Data":"892021219b9ade1b1e3cb49da7607c75daf006ee8b3d2765f9896f3f8d932187"} Jul 02 23:22:29.959112 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.958952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5db7bc6d5f-r857s" event={"ID":"f2f4c5e2-71fc-4186-a190-2b77cf025bd9","Type":"ContainerStarted","Data":"0d3667904ab51f281c8cdf2276dbbcc23673273f85df29e0281eab8105ccf86e"} Jul 02 23:22:29.977057 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:29.976996 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5db7bc6d5f-r857s" podStartSLOduration=0.646344637 podStartE2EDuration="976.97842ms" totalImagesPullingTime="330.633783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:22:29.975428356 +0000 UTC m=+724.043040916" watchObservedRunningTime="2026-07-02 23:22:29.97697842 +0000 UTC m=+724.044591031" Jul 02 23:22:58.866559 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:22:58.866523 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:23:08.563990 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:23:08.563954 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:23:19.960682 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:23:19.960638 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:23:28.371010 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:23:28.370974 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:23:39.177914 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:23:39.177877 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:24:09.769048 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:24:09.769009 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:24:18.863131 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:24:18.863096 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:24:49.666870 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:24:49.666831 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:24:56.572154 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:24:56.572112 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:06.859890 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:06.859846 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:13.085701 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:13.085666 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:23.368995 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:23.368910 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:26.412699 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:26.412671 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:25:26.414023 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:26.414004 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:25:33.463853 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:33.463810 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:44.363544 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:44.363499 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:25:53.669237 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:25:53.669199 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:26:16.770308 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:26:16.770265 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:26:25.866541 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:26:25.866494 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:26:51.563946 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:26:51.563861 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:01.059595 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:01.059553 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:07.266269 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:07.266235 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:16.770258 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:16.770204 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:26.967647 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:26.967609 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:36.361279 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:36.361233 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:46.761845 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:46.761806 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:27:55.553910 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:27:55.553866 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:29:42.359685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:29:42.359649 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:29:51.067125 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:29:51.067042 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:05.159885 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:05.159845 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:14.662181 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:14.662137 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:25.459945 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:25.459905 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:26.448702 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:26.448672 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:30:26.451761 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:26.451730 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:30:37.365336 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:37.365293 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:45.858684 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:45.858641 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:30:56.466798 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:30:56.466755 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:31:07.162483 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:31:07.162442 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:31:16.559755 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:31:16.559715 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:31:25.565369 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:31:25.565280 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:31:51.163454 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:31:51.163413 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:31:58.075988 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:31:58.075945 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:32:06.767440 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:32:06.767400 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:32:18.959601 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:32:18.959564 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:32:35.668014 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:32:35.667977 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:32:46.060511 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:32:46.060475 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:32:55.263072 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:32:55.262991 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:06.070259 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:06.070215 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:14.767772 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:14.767732 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:23.258795 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:23.258751 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:36.361293 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:36.361254 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:46.361504 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:46.361456 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:33:55.059114 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:33:55.059069 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:05.952686 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:05.952652 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:16.460868 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:16.460831 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:26.068194 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:26.068105 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:36.172778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:36.172736 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:47.362390 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:47.362352 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:34:57.377167 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:34:57.377116 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:35:10.369630 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:10.369592 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:35:18.759467 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:18.759428 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:35:26.482051 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:26.482017 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:35:26.485636 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:26.485608 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:35:28.668792 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:28.668759 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:35:45.667858 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:45.667821 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:35:56.758797 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:35:56.758763 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:36:15.964354 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:36:15.964316 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:36:26.263088 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:36:26.263053 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:37:34.970190 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:37:34.970103 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:37:42.267043 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:37:42.266994 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:37:51.067165 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:37:51.067106 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:38:02.762697 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:38:02.762657 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:38:12.256597 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:38:12.256561 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:38:38.455972 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:38:38.455933 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:38:47.365685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:38:47.365647 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:39:17.967320 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:39:17.967285 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:39:30.069213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:39:30.069174 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:40:08.958942 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:40:08.958899 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:40:26.514946 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:40:26.514917 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:40:26.523284 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:40:26.523257 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:41:15.577455 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:15.577408 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:41:31.370815 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:31.370780 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:41:45.066793 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.066740 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:41:45.070308 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.070287 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.074295 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.074264 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a4ec8d\"/\"test-model-a4ec8d-kserve-self-signed-certs\"" Jul 02 23:41:45.074295 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.074262 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a4ec8d\"/\"kube-root-ca.crt\"" Jul 02 23:41:45.074295 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.074273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a4ec8d\"/\"openshift-service-ca.crt\"" Jul 02 23:41:45.074642 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.074319 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a4ec8d\"/\"default-dockercfg-4xr5b\"" Jul 02 23:41:45.079323 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.079299 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:41:45.127320 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.127530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.127530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127420 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.127530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127444 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vltpb\" (UniqueName: \"kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.127530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.127530 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.127500 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.228993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229070 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vltpb\" (UniqueName: \"kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229180 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229213 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229197 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229287 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229394 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.229604 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.229583 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.231469 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.231442 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.231726 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.231708 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.237734 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.237706 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vltpb\" (UniqueName: \"kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb\") pod \"test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.380494 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.380396 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:45.522250 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.522225 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:41:45.523861 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:41:45.523833 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e6724cc_78d4_4f4f_8f1b_4154b5b954c7.slice/crio-1596dfc34c25b7fa79837792311fb970b85fced73f935c0205b59cce5aa0e72a WatchSource:0}: Error finding container 1596dfc34c25b7fa79837792311fb970b85fced73f935c0205b59cce5aa0e72a: Status 404 returned error can't find the container with id 1596dfc34c25b7fa79837792311fb970b85fced73f935c0205b59cce5aa0e72a Jul 02 23:41:45.930504 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.930465 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" event={"ID":"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7","Type":"ContainerStarted","Data":"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c"} Jul 02 23:41:45.930702 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.930511 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" event={"ID":"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7","Type":"ContainerStarted","Data":"1596dfc34c25b7fa79837792311fb970b85fced73f935c0205b59cce5aa0e72a"} Jul 02 23:41:45.948704 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:45.948639 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" podStartSLOduration=0.948622279 podStartE2EDuration="948.622279ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:41:45.946452462 +0000 UTC m=+1880.014065045" watchObservedRunningTime="2026-07-02 23:41:45.948622279 +0000 UTC m=+1880.016234892" Jul 02 23:41:48.264640 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:48.264555 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:41:55.380866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:55.380812 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:55.380866 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:55.380874 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:55.393574 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:55.393507 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:55.998284 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:55.998254 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:41:59.064027 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:41:59.063996 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:42:10.370075 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:10.370034 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:42:25.226922 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:25.226884 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:42:30.262929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:30.262890 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:42:40.248105 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.248042 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" podUID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" containerName="main" containerID="cri-o://8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c" gracePeriod=45 Jul 02 23:42:40.506032 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.505969 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:42:40.622100 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622057 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vltpb\" (UniqueName: \"kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622118 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622176 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622237 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622271 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622307 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622291 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm\") pod \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\" (UID: \"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7\") " Jul 02 23:42:40.622586 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622397 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:40.622586 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622476 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:40.622692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622601 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:40.622692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622607 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home" (OuterVolumeSpecName: "home") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:40.622692 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.622618 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:40.624510 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.624470 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm" (OuterVolumeSpecName: "dshm") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:42:40.624510 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.624543 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:42:40.624510 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.624561 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb" (OuterVolumeSpecName: "kube-api-access-vltpb") pod "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" (UID: "5e6724cc-78d4-4f4f-8f1b-4154b5b954c7"). InnerVolumeSpecName "kube-api-access-vltpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:42:40.723575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.723494 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:40.723575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.723568 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:40.723575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.723579 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:40.723575 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:40.723587 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vltpb\" (UniqueName: \"kubernetes.io/projected/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7-kube-api-access-vltpb\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:42:41.216641 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.216603 2558 generic.go:356] "Generic (PLEG): container finished" podID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" containerID="8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c" exitCode=0 Jul 02 23:42:41.216838 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.216651 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" Jul 02 23:42:41.216838 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.216682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" event={"ID":"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7","Type":"ContainerDied","Data":"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c"} Jul 02 23:42:41.216838 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.216724 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb" event={"ID":"5e6724cc-78d4-4f4f-8f1b-4154b5b954c7","Type":"ContainerDied","Data":"1596dfc34c25b7fa79837792311fb970b85fced73f935c0205b59cce5aa0e72a"} Jul 02 23:42:41.216838 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.216741 2558 scope.go:117] "RemoveContainer" containerID="8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c" Jul 02 23:42:41.226496 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.226473 2558 scope.go:117] "RemoveContainer" containerID="8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c" Jul 02 23:42:41.226803 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:42:41.226781 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c\": container with ID starting with 8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c not found: ID does not exist" containerID="8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c" Jul 02 23:42:41.226874 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.226816 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c"} err="failed to get container status \"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c\": rpc error: code = NotFound desc = could not find container \"8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c\": container with ID starting with 8cf3b32392c3577da5c2e7e9f65c3bfffb14fa6b9c0a1728861b877f378e148c not found: ID does not exist" Jul 02 23:42:41.239362 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.239332 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:42:41.242726 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:41.242701 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a4ec8d/test-model-a4ec8d-kserve-6b8c98f5ff-rhfqb"] Jul 02 23:42:42.444867 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:42:42.444835 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" path="/var/lib/kubelet/pods/5e6724cc-78d4-4f4f-8f1b-4154b5b954c7/volumes" Jul 02 23:43:12.964544 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:43:12.964496 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:43:25.158343 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:43:25.158253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:43:37.481891 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:43:37.481854 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:43:52.097912 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:43:52.097868 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:44:03.044233 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:03.044196 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:44:18.168651 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:18.168619 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:44:25.764081 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.764047 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:44:25.764876 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.764852 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" containerName="main" Jul 02 23:44:25.764876 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.764878 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" containerName="main" Jul 02 23:44:25.765038 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.764954 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6724cc-78d4-4f4f-8f1b-4154b5b954c7" containerName="main" Jul 02 23:44:25.768260 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.768231 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.772206 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.772169 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"auth-test-model-aefe3e-kserve-self-signed-certs\"" Jul 02 23:44:25.772344 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.772238 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"default-dockercfg-kmgbx\"" Jul 02 23:44:25.772344 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.772174 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"openshift-service-ca.crt\"" Jul 02 23:44:25.772344 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.772303 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"kube-root-ca.crt\"" Jul 02 23:44:25.778078 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.778046 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:44:25.828707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828658 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.828707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.828924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828768 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.828924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828807 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8whg9\" (UniqueName: \"kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.828924 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828888 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.829025 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.828927 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930052 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930007 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930265 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930065 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930265 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930265 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930265 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930135 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930265 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8whg9\" (UniqueName: \"kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930607 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930577 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930677 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930623 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.930677 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.930641 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.932483 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.932455 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.932789 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.932765 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:25.938143 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:25.938122 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8whg9\" (UniqueName: \"kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9\") pod \"auth-test-model-aefe3e-kserve-766c7f747-cmrsb\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:26.079348 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:26.079244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:26.218409 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:26.218373 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:44:26.222001 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:44:26.221971 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2338fcf_b8ea_4908_9713_027ecf4fc076.slice/crio-7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a WatchSource:0}: Error finding container 7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a: Status 404 returned error can't find the container with id 7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a Jul 02 23:44:26.750839 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:26.750800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" event={"ID":"d2338fcf-b8ea-4908-9713-027ecf4fc076","Type":"ContainerStarted","Data":"de40942fedd32a53e91372a195e801a1b606ebc038e384b150da30efdafc0e24"} Jul 02 23:44:26.750839 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:26.750850 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" event={"ID":"d2338fcf-b8ea-4908-9713-027ecf4fc076","Type":"ContainerStarted","Data":"7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a"} Jul 02 23:44:26.768565 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:26.768474 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" podStartSLOduration=1.768457237 podStartE2EDuration="1.768457237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:44:26.767086144 +0000 UTC m=+2040.834698704" watchObservedRunningTime="2026-07-02 23:44:26.768457237 +0000 UTC m=+2040.836069844" Jul 02 23:44:30.472736 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:30.472699 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:44:36.080155 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:36.080107 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:36.080155 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:36.080158 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:36.093352 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:36.093323 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:36.812505 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:36.812467 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:44:56.367032 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:44:56.366932 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:45:14.365689 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:14.365650 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:45:26.551155 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:26.551125 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:45:26.559397 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:26.559372 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:45:29.666568 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:29.666505 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:45:44.359985 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:44.359940 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:45:59.866902 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:45:59.866867 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:46:17.758876 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:17.758838 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:46:35.059107 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:35.059006 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:46:48.294811 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:48.294774 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:46:51.669305 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:51.669268 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:46:56.568460 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.568422 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:46:56.572206 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.572167 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.574696 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.574666 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"sub-test-model-aefe3e-kserve-self-signed-certs\"" Jul 02 23:46:56.580335 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.580302 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:46:56.682382 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682346 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.682609 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.682609 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.682609 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682496 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.682609 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682543 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tbdt\" (UniqueName: \"kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.682609 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.682582 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783709 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783667 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783768 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783810 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783837 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tbdt\" (UniqueName: \"kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.783901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.783889 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.784162 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.784129 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.784221 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.784166 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.784273 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.784222 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.786164 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.786136 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.786474 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.786454 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.792479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.792451 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tbdt\" (UniqueName: \"kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt\") pod \"sub-test-model-aefe3e-kserve-9664b5869-w8gv5\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:56.882611 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:56.882481 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:46:57.023974 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:57.023947 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:46:57.025302 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:46:57.025275 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5152b2a1_5bfb_43eb_9584_7efb02382d11.slice/crio-624e86b52a73a16d4fe91c520280c0c6f79754d46e2112047c08eadd6bdb54e8 WatchSource:0}: Error finding container 624e86b52a73a16d4fe91c520280c0c6f79754d46e2112047c08eadd6bdb54e8: Status 404 returned error can't find the container with id 624e86b52a73a16d4fe91c520280c0c6f79754d46e2112047c08eadd6bdb54e8 Jul 02 23:46:57.535165 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:57.535129 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" event={"ID":"5152b2a1-5bfb-43eb-9584-7efb02382d11","Type":"ContainerStarted","Data":"f914543615947b301d66611353ef3f695fd6a7d607f2994f021e5adeb1fd9ae9"} Jul 02 23:46:57.535165 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:57.535172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" event={"ID":"5152b2a1-5bfb-43eb-9584-7efb02382d11","Type":"ContainerStarted","Data":"624e86b52a73a16d4fe91c520280c0c6f79754d46e2112047c08eadd6bdb54e8"} Jul 02 23:46:57.556713 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:46:57.556641 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" podStartSLOduration=1.556620079 podStartE2EDuration="1.556620079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:46:57.551021685 +0000 UTC m=+2191.618634244" watchObservedRunningTime="2026-07-02 23:46:57.556620079 +0000 UTC m=+2191.624232639" Jul 02 23:47:03.315068 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.315005 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" podUID="d2338fcf-b8ea-4908-9713-027ecf4fc076" containerName="main" containerID="cri-o://de40942fedd32a53e91372a195e801a1b606ebc038e384b150da30efdafc0e24" gracePeriod=45 Jul 02 23:47:03.572814 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.572725 2558 generic.go:356] "Generic (PLEG): container finished" podID="d2338fcf-b8ea-4908-9713-027ecf4fc076" containerID="de40942fedd32a53e91372a195e801a1b606ebc038e384b150da30efdafc0e24" exitCode=0 Jul 02 23:47:03.572958 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.572800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" event={"ID":"d2338fcf-b8ea-4908-9713-027ecf4fc076","Type":"ContainerDied","Data":"de40942fedd32a53e91372a195e801a1b606ebc038e384b150da30efdafc0e24"} Jul 02 23:47:03.572958 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.572847 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" event={"ID":"d2338fcf-b8ea-4908-9713-027ecf4fc076","Type":"ContainerDied","Data":"7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a"} Jul 02 23:47:03.572958 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.572864 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d766d499f92a8b788ee242a55afadb6efcce98421772afe5f9fce4506a6e63a" Jul 02 23:47:03.576224 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.576199 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:47:03.643146 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643101 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643194 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643220 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643267 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8whg9\" (UniqueName: \"kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643334 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643330 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643545 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643382 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs\") pod \"d2338fcf-b8ea-4908-9713-027ecf4fc076\" (UID: \"d2338fcf-b8ea-4908-9713-027ecf4fc076\") " Jul 02 23:47:03.643545 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643414 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:47:03.643545 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643466 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:47:03.643704 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643634 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home" (OuterVolumeSpecName: "home") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:47:03.643704 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643684 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:03.643704 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.643699 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:03.645764 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.645727 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm" (OuterVolumeSpecName: "dshm") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:47:03.645881 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.645783 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:47:03.645881 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.645809 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9" (OuterVolumeSpecName: "kube-api-access-8whg9") pod "d2338fcf-b8ea-4908-9713-027ecf4fc076" (UID: "d2338fcf-b8ea-4908-9713-027ecf4fc076"). InnerVolumeSpecName "kube-api-access-8whg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:47:03.744057 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.744017 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2338fcf-b8ea-4908-9713-027ecf4fc076-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:03.744057 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.744049 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:03.744057 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.744062 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8whg9\" (UniqueName: \"kubernetes.io/projected/d2338fcf-b8ea-4908-9713-027ecf4fc076-kube-api-access-8whg9\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:03.744294 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:03.744077 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2338fcf-b8ea-4908-9713-027ecf4fc076-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:47:04.577990 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:04.577960 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb" Jul 02 23:47:04.594428 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:04.594390 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:47:04.598327 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:04.598288 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/auth-test-model-aefe3e-kserve-766c7f747-cmrsb"] Jul 02 23:47:05.764643 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:05.764607 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:47:06.445447 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:06.445406 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2338fcf-b8ea-4908-9713-027ecf4fc076" path="/var/lib/kubelet/pods/d2338fcf-b8ea-4908-9713-027ecf4fc076/volumes" Jul 02 23:47:06.882705 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:06.882662 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:47:06.883132 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:06.882725 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:47:06.895277 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:06.895244 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:47:07.611137 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:07.611100 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:47:20.567910 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:20.567873 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:47:44.170593 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:47:44.170555 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:48:00.961205 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:00.961167 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:48:07.775098 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:07.775063 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:48:18.770402 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.770369 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:48:18.770883 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.770726 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2338fcf-b8ea-4908-9713-027ecf4fc076" containerName="main" Jul 02 23:48:18.770883 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.770737 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2338fcf-b8ea-4908-9713-027ecf4fc076" containerName="main" Jul 02 23:48:18.770883 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.770809 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2338fcf-b8ea-4908-9713-027ecf4fc076" containerName="main" Jul 02 23:48:18.773762 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.773745 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.776551 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.776529 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aefe3e\"/\"rate-test-model-aefe3e-kserve-self-signed-certs\"" Jul 02 23:48:18.785015 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.784985 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:48:18.901305 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901263 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.901487 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901345 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5bh9\" (UniqueName: \"kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.901487 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901379 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.901487 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.901655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:18.901655 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:18.901642 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002220 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002220 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002223 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bh9\" (UniqueName: \"kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002564 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002243 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002564 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002277 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002564 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002293 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002564 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002482 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002737 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002688 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002776 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002740 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.002776 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.002761 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.004741 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.004717 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.004901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.004885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.011561 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.011508 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5bh9\" (UniqueName: \"kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9\") pod \"rate-test-model-aefe3e-kserve-78bc679d5f-q79j5\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.082678 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.082579 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:19.226299 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.226256 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:48:19.227691 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:48:19.227664 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87ecf7c3_a42d_48fc_b1e8_09225d4872f1.slice/crio-4204ed1e113be91c082d1154219c486c7dd6befd1ac406c785d91876b09581fd WatchSource:0}: Error finding container 4204ed1e113be91c082d1154219c486c7dd6befd1ac406c785d91876b09581fd: Status 404 returned error can't find the container with id 4204ed1e113be91c082d1154219c486c7dd6befd1ac406c785d91876b09581fd Jul 02 23:48:19.358304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.358209 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:48:19.984653 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.984601 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" event={"ID":"87ecf7c3-a42d-48fc-b1e8-09225d4872f1","Type":"ContainerStarted","Data":"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467"} Jul 02 23:48:19.984653 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:19.984660 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" event={"ID":"87ecf7c3-a42d-48fc-b1e8-09225d4872f1","Type":"ContainerStarted","Data":"4204ed1e113be91c082d1154219c486c7dd6befd1ac406c785d91876b09581fd"} Jul 02 23:48:20.027172 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:20.027113 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" podStartSLOduration=2.027094633 podStartE2EDuration="2.027094633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:48:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:48:20.025670483 +0000 UTC m=+2274.093283043" watchObservedRunningTime="2026-07-02 23:48:20.027094633 +0000 UTC m=+2274.094707192" Jul 02 23:48:22.794868 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:22.794823 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" podUID="5152b2a1-5bfb-43eb-9584-7efb02382d11" containerName="main" containerID="cri-o://f914543615947b301d66611353ef3f695fd6a7d607f2994f021e5adeb1fd9ae9" gracePeriod=45 Jul 02 23:48:23.001951 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.001918 2558 generic.go:356] "Generic (PLEG): container finished" podID="5152b2a1-5bfb-43eb-9584-7efb02382d11" containerID="f914543615947b301d66611353ef3f695fd6a7d607f2994f021e5adeb1fd9ae9" exitCode=0 Jul 02 23:48:23.001951 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.001957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" event={"ID":"5152b2a1-5bfb-43eb-9584-7efb02382d11","Type":"ContainerDied","Data":"f914543615947b301d66611353ef3f695fd6a7d607f2994f021e5adeb1fd9ae9"} Jul 02 23:48:23.046691 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.046627 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:48:23.141695 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141658 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.141882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141721 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.141882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141763 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.141882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141795 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.141882 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141866 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.142105 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.141898 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tbdt\" (UniqueName: \"kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt\") pod \"5152b2a1-5bfb-43eb-9584-7efb02382d11\" (UID: \"5152b2a1-5bfb-43eb-9584-7efb02382d11\") " Jul 02 23:48:23.142105 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142077 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home" (OuterVolumeSpecName: "home") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:23.142203 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142117 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache" (OuterVolumeSpecName: "model-cache") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:23.142203 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142130 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:23.142567 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142229 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:23.142567 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142249 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:23.142567 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.142265 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:23.143886 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.143861 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm" (OuterVolumeSpecName: "dshm") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:23.143942 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.143895 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt" (OuterVolumeSpecName: "kube-api-access-2tbdt") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "kube-api-access-2tbdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:48:23.143979 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.143950 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5152b2a1-5bfb-43eb-9584-7efb02382d11" (UID: "5152b2a1-5bfb-43eb-9584-7efb02382d11"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:48:23.243661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.243619 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5152b2a1-5bfb-43eb-9584-7efb02382d11-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:23.243661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.243650 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tbdt\" (UniqueName: \"kubernetes.io/projected/5152b2a1-5bfb-43eb-9584-7efb02382d11-kube-api-access-2tbdt\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:23.243661 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:23.243660 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5152b2a1-5bfb-43eb-9584-7efb02382d11-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:48:24.008468 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.008439 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" Jul 02 23:48:24.008468 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.008448 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5" event={"ID":"5152b2a1-5bfb-43eb-9584-7efb02382d11","Type":"ContainerDied","Data":"624e86b52a73a16d4fe91c520280c0c6f79754d46e2112047c08eadd6bdb54e8"} Jul 02 23:48:24.008966 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.008498 2558 scope.go:117] "RemoveContainer" containerID="f914543615947b301d66611353ef3f695fd6a7d607f2994f021e5adeb1fd9ae9" Jul 02 23:48:24.037223 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.037189 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:48:24.040380 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.040351 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/sub-test-model-aefe3e-kserve-9664b5869-w8gv5"] Jul 02 23:48:24.447147 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:24.447071 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5152b2a1-5bfb-43eb-9584-7efb02382d11" path="/var/lib/kubelet/pods/5152b2a1-5bfb-43eb-9584-7efb02382d11/volumes" Jul 02 23:48:29.082742 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:29.082696 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:29.082742 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:29.082752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:29.095296 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:29.095267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:30.053479 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:30.053444 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:48:30.565987 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:30.565953 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:48:46.565598 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:48:46.565557 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:49:01.668071 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:01.668032 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:49:18.869197 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:18.869083 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:49:22.693172 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:22.693105 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:49:33.461752 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:33.461709 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:49:37.713812 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:37.713746 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" podUID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" containerName="main" containerID="cri-o://e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467" gracePeriod=45 Jul 02 23:49:37.971733 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:37.971669 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:49:38.083048 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083002 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083048 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083050 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083100 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083133 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5bh9\" (UniqueName: \"kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083195 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083298 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083223 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs\") pod \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\" (UID: \"87ecf7c3-a42d-48fc-b1e8-09225d4872f1\") " Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083306 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083349 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home" (OuterVolumeSpecName: "home") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083433 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083546 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083580 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.083596 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.083589 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.085354 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.085328 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm" (OuterVolumeSpecName: "dshm") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:38.085481 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.085334 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:49:38.085481 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.085360 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9" (OuterVolumeSpecName: "kube-api-access-k5bh9") pod "87ecf7c3-a42d-48fc-b1e8-09225d4872f1" (UID: "87ecf7c3-a42d-48fc-b1e8-09225d4872f1"). InnerVolumeSpecName "kube-api-access-k5bh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:38.184602 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.184564 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.184602 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.184596 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.184602 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.184606 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5bh9\" (UniqueName: \"kubernetes.io/projected/87ecf7c3-a42d-48fc-b1e8-09225d4872f1-kube-api-access-k5bh9\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\"" Jul 02 23:49:38.412914 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.412878 2558 generic.go:356] "Generic (PLEG): container finished" podID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" containerID="e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467" exitCode=0 Jul 02 23:49:38.413163 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.412921 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" Jul 02 23:49:38.413163 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.412965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" event={"ID":"87ecf7c3-a42d-48fc-b1e8-09225d4872f1","Type":"ContainerDied","Data":"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467"} Jul 02 23:49:38.413163 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.413010 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5" event={"ID":"87ecf7c3-a42d-48fc-b1e8-09225d4872f1","Type":"ContainerDied","Data":"4204ed1e113be91c082d1154219c486c7dd6befd1ac406c785d91876b09581fd"} Jul 02 23:49:38.413163 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.413032 2558 scope.go:117] "RemoveContainer" containerID="e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467" Jul 02 23:49:38.422339 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.422319 2558 scope.go:117] "RemoveContainer" containerID="e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467" Jul 02 23:49:38.422639 ip-10-0-135-240 kubenswrapper[2558]: E0702 23:49:38.422618 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467\": container with ID starting with e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467 not found: ID does not exist" containerID="e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467" Jul 02 23:49:38.422703 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.422685 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467"} err="failed to get container status \"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467\": rpc error: code = NotFound desc = could not find container \"e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467\": container with ID starting with e7224d5e009e5f871d93d3e30700f202df0c1dbf4c602e2526eb75f26e957467 not found: ID does not exist" Jul 02 23:49:38.434206 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.434179 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:49:38.445221 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:38.445197 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aefe3e/rate-test-model-aefe3e-kserve-78bc679d5f-q79j5"] Jul 02 23:49:40.445246 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:49:40.445210 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" path="/var/lib/kubelet/pods/87ecf7c3-a42d-48fc-b1e8-09225d4872f1/volumes" Jul 02 23:50:10.071819 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:10.071785 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-g4htq"] Jul 02 23:50:14.780689 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:14.780659 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5db7bc6d5f-r857s_f2f4c5e2-71fc-4186-a190-2b77cf025bd9/authorino/0.log" Jul 02 23:50:19.955107 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:19.955078 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j866p_11eb27b6-abbc-4520-a929-2c29023ab3ce/server/0.log" Jul 02 23:50:20.182023 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:20.181993 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9978976-4xmzq_90c768c9-e1d6-48f3-8cbb-8b18973367dd/copy-manifests/0.log" Jul 02 23:50:20.188979 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:20.188954 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9978976-4xmzq_90c768c9-e1d6-48f3-8cbb-8b18973367dd/manager/0.log" Jul 02 23:50:21.288834 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.288802 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/pull/0.log" Jul 02 23:50:21.295153 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.295126 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/extract/0.log" Jul 02 23:50:21.301825 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.301798 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/util/0.log" Jul 02 23:50:21.414510 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.414478 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/pull/0.log" Jul 02 23:50:21.420840 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.420817 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/extract/0.log" Jul 02 23:50:21.427224 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.427202 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/util/0.log" Jul 02 23:50:21.536206 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.536169 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/util/0.log" Jul 02 23:50:21.543844 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.543763 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/pull/0.log" Jul 02 23:50:21.550778 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.550755 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/extract/0.log" Jul 02 23:50:21.666399 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.666371 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/util/0.log" Jul 02 23:50:21.673950 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.673923 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/pull/0.log" Jul 02 23:50:21.680960 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.680937 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/extract/0.log" Jul 02 23:50:21.792485 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:21.792454 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5db7bc6d5f-r857s_f2f4c5e2-71fc-4186-a190-2b77cf025bd9/authorino/0.log" Jul 02 23:50:22.484629 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:22.484595 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-g4htq_43655c91-5405-4775-9666-e909ba94e8f7/limitador/0.log" Jul 02 23:50:23.294603 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:23.294570 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69478b7d6-xvwsg_961e27db-bde4-4e3d-b15a-e93e125061f0/kube-auth-proxy/0.log" Jul 02 23:50:23.754921 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:23.754888 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-784c48b49-hfwdq_f51c65fa-18ca-426b-9beb-c15a20ce99e5/router/0.log" Jul 02 23:50:24.409275 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:24.409247 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbq9wxx_e1a5f2cf-ed6e-4f27-9bad-2ac4100ee850/main/0.log" Jul 02 23:50:24.624061 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:24.624030 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-5xghx_d6a5a43c-b566-4b4f-a09d-f015121e8631/main/0.log" Jul 02 23:50:26.587241 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:26.587204 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:50:26.597180 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:26.597137 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:50:26.632983 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:26.632949 2558 scope.go:117] "RemoveContainer" containerID="de40942fedd32a53e91372a195e801a1b606ebc038e384b150da30efdafc0e24" Jul 02 23:50:32.049978 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:32.049944 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2d76z_9467f990-5431-4897-860b-e938c0872949/global-pull-secret-syncer/0.log" Jul 02 23:50:32.300598 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:32.300508 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xmb22_ff88c77e-21fb-401f-9c2f-9c508f3270c5/konnectivity-agent/0.log" Jul 02 23:50:32.360569 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:32.360539 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-240.ec2.internal_3b12c154aed0715acfb2aca1c887480a/haproxy/0.log" Jul 02 23:50:36.392658 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.392629 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/extract/0.log" Jul 02 23:50:36.416543 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.416500 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/util/0.log" Jul 02 23:50:36.447175 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.447145 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d67s6_e116172e-3312-424d-a776-4d1cecba1e72/pull/0.log" Jul 02 23:50:36.483339 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.483312 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/extract/0.log" Jul 02 23:50:36.509304 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.509271 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/util/0.log" Jul 02 23:50:36.535473 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.535448 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0d9b7g_04ae7ee9-1667-4dee-b7a8-e9169daa4928/pull/0.log" Jul 02 23:50:36.562662 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.562637 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/extract/0.log" Jul 02 23:50:36.587928 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.587903 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/util/0.log" Jul 02 23:50:36.611128 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.611098 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737krw7_06fa0b0b-0822-446c-a354-0e2b71a2154c/pull/0.log" Jul 02 23:50:36.646335 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.646246 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/extract/0.log" Jul 02 23:50:36.679384 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.679355 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/util/0.log" Jul 02 23:50:36.707929 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.707897 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cwm5n_8cfce3e3-46a5-465f-a9a4-b3e492a0cc94/pull/0.log" Jul 02 23:50:36.988444 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:36.983783 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5db7bc6d5f-r857s_f2f4c5e2-71fc-4186-a190-2b77cf025bd9/authorino/0.log" Jul 02 23:50:37.424543 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:37.424432 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-g4htq_43655c91-5405-4775-9666-e909ba94e8f7/limitador/0.log" Jul 02 23:50:39.129805 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:39.129768 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-bgp5m_f3d78fe5-69c9-4854-ae87-82e7e4719dc0/cluster-monitoring-operator/0.log" Jul 02 23:50:39.433601 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:39.433499 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9psw9_2d56d0c7-5897-4af0-ba44-88013b0f34c5/node-exporter/0.log" Jul 02 23:50:39.462478 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:39.462452 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9psw9_2d56d0c7-5897-4af0-ba44-88013b0f34c5/kube-rbac-proxy/0.log" Jul 02 23:50:39.489450 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:39.489425 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9psw9_2d56d0c7-5897-4af0-ba44-88013b0f34c5/init-textfile/0.log" Jul 02 23:50:41.416902 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:41.416869 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-mjkj2_62a2b086-a180-428e-abce-6516c0c1d001/networking-console-plugin/0.log" Jul 02 23:50:42.030211 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.030170 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/1.log" Jul 02 23:50:42.035141 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.035116 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-q62ks_af98f986-4915-48d1-80fa-4a8a11b99dc1/console-operator/2.log" Jul 02 23:50:42.175401 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175371 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw"] Jul 02 23:50:42.175884 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175836 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5152b2a1-5bfb-43eb-9584-7efb02382d11" containerName="main" Jul 02 23:50:42.175884 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175859 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5152b2a1-5bfb-43eb-9584-7efb02382d11" containerName="main" Jul 02 23:50:42.175884 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175872 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" containerName="main" Jul 02 23:50:42.175884 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175880 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" containerName="main" Jul 02 23:50:42.176192 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175978 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5152b2a1-5bfb-43eb-9584-7efb02382d11" containerName="main" Jul 02 23:50:42.176192 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.175988 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ecf7c3-a42d-48fc-b1e8-09225d4872f1" containerName="main" Jul 02 23:50:42.178953 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.178933 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.181707 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.181686 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d4ctc\"/\"kube-root-ca.crt\"" Jul 02 23:50:42.181816 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.181737 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d4ctc\"/\"openshift-service-ca.crt\"" Jul 02 23:50:42.182894 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.182876 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d4ctc\"/\"default-dockercfg-jvzp8\"" Jul 02 23:50:42.186430 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.186407 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw"] Jul 02 23:50:42.235431 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.235371 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-lib-modules\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.235431 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.235425 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dckl5\" (UniqueName: \"kubernetes.io/projected/f12783b5-1491-451e-9ed6-f52e01581128-kube-api-access-dckl5\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.235685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.235590 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-sys\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.235685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.235623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-proc\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.235685 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.235659 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-podres\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336627 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336538 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-lib-modules\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336627 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336580 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dckl5\" (UniqueName: \"kubernetes.io/projected/f12783b5-1491-451e-9ed6-f52e01581128-kube-api-access-dckl5\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-sys\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336641 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-lib-modules\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336649 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-proc\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336704 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-proc\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-podres\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-sys\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.336823 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.336775 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f12783b5-1491-451e-9ed6-f52e01581128-podres\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.344854 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.344820 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dckl5\" (UniqueName: \"kubernetes.io/projected/f12783b5-1491-451e-9ed6-f52e01581128-kube-api-access-dckl5\") pod \"perf-node-gather-daemonset-7d5xw\" (UID: \"f12783b5-1491-451e-9ed6-f52e01581128\") " pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.490154 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.490115 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.624391 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.624348 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw"] Jul 02 23:50:42.624901 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.624879 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-tx54n_11583f25-c965-40a0-a021-cd8a433a1834/download-server/0.log" Jul 02 23:50:42.626888 ip-10-0-135-240 kubenswrapper[2558]: W0702 23:50:42.626861 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf12783b5_1491_451e_9ed6_f52e01581128.slice/crio-1a6f0a021af6adf0e987e27720d3a5b1e29fe902af4159c6694fefd9f4a4c727 WatchSource:0}: Error finding container 1a6f0a021af6adf0e987e27720d3a5b1e29fe902af4159c6694fefd9f4a4c727: Status 404 returned error can't find the container with id 1a6f0a021af6adf0e987e27720d3a5b1e29fe902af4159c6694fefd9f4a4c727 Jul 02 23:50:42.743848 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.743811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" event={"ID":"f12783b5-1491-451e-9ed6-f52e01581128","Type":"ContainerStarted","Data":"1de19993e46e64cee3ef09b57c24de18de502428a2892809306aa7e39a6bf95b"} Jul 02 23:50:42.744018 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.743866 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" event={"ID":"f12783b5-1491-451e-9ed6-f52e01581128","Type":"ContainerStarted","Data":"1a6f0a021af6adf0e987e27720d3a5b1e29fe902af4159c6694fefd9f4a4c727"} Jul 02 23:50:42.744018 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.743895 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:42.761268 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:42.761214 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" podStartSLOduration=0.761195097 podStartE2EDuration="761.195097ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:50:42.759644198 +0000 UTC m=+2416.827256757" watchObservedRunningTime="2026-07-02 23:50:42.761195097 +0000 UTC m=+2416.828807656" Jul 02 23:50:43.221424 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:43.221395 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-dk7wh_116d4405-27af-41ac-a5a3-b13cffd78d7b/volume-data-source-validator/0.log" Jul 02 23:50:44.335952 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:44.335921 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmmhq_b406076e-94af-44ac-97c0-e2eba90effa4/dns/0.log" Jul 02 23:50:44.357999 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:44.357970 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rmmhq_b406076e-94af-44ac-97c0-e2eba90effa4/kube-rbac-proxy/0.log" Jul 02 23:50:44.439841 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:44.439810 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xltcn_f7603a47-9ef4-4211-adcc-1c31975fc699/dns-node-resolver/0.log" Jul 02 23:50:45.049796 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:45.049769 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tk6cr_d34d323b-43bd-4ffc-8818-e2d460f0f7be/node-ca/0.log" Jul 02 23:50:46.151618 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:46.151582 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69478b7d6-xvwsg_961e27db-bde4-4e3d-b15a-e93e125061f0/kube-auth-proxy/0.log" Jul 02 23:50:46.321384 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:46.321357 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-784c48b49-hfwdq_f51c65fa-18ca-426b-9beb-c15a20ce99e5/router/0.log" Jul 02 23:50:46.998173 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:46.998145 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kz2k8_d23b8783-dbea-4420-98a5-85a56ff2e7a9/serve-healthcheck-canary/0.log" Jul 02 23:50:47.519029 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:47.518994 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-w4sdf_da583992-cc89-437b-a199-67b2a476ad7a/insights-operator/0.log" Jul 02 23:50:47.520053 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:47.520036 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-w4sdf_da583992-cc89-437b-a199-67b2a476ad7a/insights-operator/1.log" Jul 02 23:50:47.610573 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:47.610490 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7kjpw_4b4aac5f-0e36-44c7-aaa5-0629e14e170c/kube-rbac-proxy/0.log" Jul 02 23:50:47.631842 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:47.631817 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7kjpw_4b4aac5f-0e36-44c7-aaa5-0629e14e170c/exporter/0.log" Jul 02 23:50:47.656785 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:47.656761 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7kjpw_4b4aac5f-0e36-44c7-aaa5-0629e14e170c/extractor/0.log" Jul 02 23:50:48.759584 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:48.759496 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d4ctc/perf-node-gather-daemonset-7d5xw" Jul 02 23:50:50.197728 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:50.197702 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-j866p_11eb27b6-abbc-4520-a929-2c29023ab3ce/server/0.log" Jul 02 23:50:50.319548 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:50.319500 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9978976-4xmzq_90c768c9-e1d6-48f3-8cbb-8b18973367dd/manager/0.log" Jul 02 23:50:50.342495 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:50.342466 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fc9978976-4xmzq_90c768c9-e1d6-48f3-8cbb-8b18973367dd/copy-manifests/0.log" Jul 02 23:50:52.009028 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:52.008999 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9d6d7b94f-j8b6c_c9b6c5fc-a483-4981-a4e3-165d975efc63/manager/0.log" Jul 02 23:50:52.692578 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:52.692488 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69478b7d6-xvwsg_961e27db-bde4-4e3d-b15a-e93e125061f0/kube-auth-proxy/0.log" Jul 02 23:50:52.824346 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:52.824308 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-784c48b49-hfwdq_f51c65fa-18ca-426b-9beb-c15a20ce99e5/router/0.log" Jul 02 23:50:54.138992 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:54.138957 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69478b7d6-xvwsg_961e27db-bde4-4e3d-b15a-e93e125061f0/kube-auth-proxy/0.log" Jul 02 23:50:54.275462 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:54.275432 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-784c48b49-hfwdq_f51c65fa-18ca-426b-9beb-c15a20ce99e5/router/0.log" Jul 02 23:50:57.519139 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:57.519105 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-m5qfj_9fb41a52-c6bd-495a-9315-1173670e2b58/kube-storage-version-migrator-operator/1.log" Jul 02 23:50:57.519928 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:57.519911 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-m5qfj_9fb41a52-c6bd-495a-9315-1173670e2b58/kube-storage-version-migrator-operator/0.log" Jul 02 23:50:58.919269 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:58.919238 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/kube-multus-additional-cni-plugins/0.log" Jul 02 23:50:58.943539 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:58.943482 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/egress-router-binary-copy/0.log" Jul 02 23:50:58.971424 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:58.971393 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/cni-plugins/0.log" Jul 02 23:50:58.993766 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:58.993737 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/bond-cni-plugin/0.log" Jul 02 23:50:59.021770 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:59.021741 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/routeoverride-cni/0.log" Jul 02 23:50:59.050725 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:59.050692 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/whereabouts-cni-bincopy/0.log" Jul 02 23:50:59.074272 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:59.074243 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zwm4l_168a5fe3-ba5c-4904-82ee-1970186c97a7/whereabouts-cni/0.log" Jul 02 23:50:59.155824 ip-10-0-135-240 kubenswrapper[2558]: I0702 23:50:59.155787 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jwbn8_a9b5cf0a-9252-4ce6-b876-32149da092bd/kube-multus/0.log"