Jul 06 14:22:45.102633 ip-10-0-141-213 systemd[1]: Starting Kubernetes Kubelet... Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:22:45.524524 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.521612 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526010 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526023 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526026 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526029 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526032 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526035 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526039 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526042 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526045 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526047 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526052 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526056 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526059 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526062 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526065 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526068 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526071 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526079 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526081 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.540407 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526084 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526087 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526090 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526092 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526095 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526097 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526100 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526104 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526108 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526111 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526114 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526116 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526119 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526122 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526125 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526127 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526130 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526133 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526135 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.547436 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526138 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526141 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526143 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526146 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526148 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526151 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526154 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526156 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526159 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526161 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526165 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526168 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526170 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526173 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526176 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526179 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526182 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526184 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526186 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526189 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.549195 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526191 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526195 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526198 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526201 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526203 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526206 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526209 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526211 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526214 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526218 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526222 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526226 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526230 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526232 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526236 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526238 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526241 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526245 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526248 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526251 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.551377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526253 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526256 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526258 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526262 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526264 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526267 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526270 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526272 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526275 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526277 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526280 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526283 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526285 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526288 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526290 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526292 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526295 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526298 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526300 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.554974 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526303 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.526305 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527769 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527776 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527780 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527783 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527786 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527789 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527792 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527796 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527799 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527802 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527805 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527807 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527810 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527814 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527818 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527821 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527824 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.557437 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527827 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527830 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527833 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527836 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527839 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527842 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527845 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527847 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527850 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527852 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527855 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527857 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527860 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527862 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527865 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527867 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527870 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527872 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527875 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527881 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.559311 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527883 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527886 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527889 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527892 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527895 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527897 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527899 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527902 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527905 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527908 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527910 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527914 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527917 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527920 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527922 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527924 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527927 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527929 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527932 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527935 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:45.561518 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527937 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527940 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527943 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527945 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527948 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527950 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527952 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527955 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527958 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527961 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527963 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527968 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527971 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527975 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527978 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527981 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527983 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527985 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527988 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527990 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:45.574732 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527992 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527995 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.527999 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528002 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528005 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528007 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528010 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528012 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528015 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528018 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528020 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528022 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528025 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528027 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528030 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528033 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528035 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528038 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528041 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.577137 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528045 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528048 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528051 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528455 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528462 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528465 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528468 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528471 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528474 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528477 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528479 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528483 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528485 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528488 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528490 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528493 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528497 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528500 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528502 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528505 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.578916 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528508 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528510 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528513 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528516 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528518 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528521 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528523 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528526 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528530 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528534 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528537 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528540 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528542 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528545 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528547 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528551 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528556 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528558 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528561 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.581044 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528564 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528567 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528570 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528572 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528575 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528577 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528580 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528583 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528586 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528590 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528592 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528595 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528597 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528599 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528602 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528605 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528607 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528610 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528612 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528615 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:45.594961 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528618 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528620 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528622 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528625 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528627 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528630 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528632 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528636 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528654 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528659 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528663 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528667 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528671 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528674 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528677 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528680 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528683 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528685 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528688 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528690 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:45.597582 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528693 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528696 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528699 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528701 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528704 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528706 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528709 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528711 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528714 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528717 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528720 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528723 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528726 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528729 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528731 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528734 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528736 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528739 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528742 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528744 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.796025 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528746 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.600557 ip-10-0-141-213 systemd[1]: Started Kubernetes Kubelet. Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528750 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.528752 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528829 2552 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528839 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528849 2552 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528855 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528861 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528866 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528873 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528882 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528885 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528888 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528893 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528897 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528899 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528902 2552 flags.go:64] FLAG: --cgroup-root="" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528905 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528908 2552 flags.go:64] FLAG: --client-ca-file="" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528911 2552 flags.go:64] FLAG: --cloud-provider="external" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528914 2552 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528919 2552 flags.go:64] FLAG: --cluster-domain="" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528921 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 14:22:45.808808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528924 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528928 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528931 2552 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528935 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528938 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528941 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528944 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528947 2552 flags.go:64] FLAG: --contention-profiling="false" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528950 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528953 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528957 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528960 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528964 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528967 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528970 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528974 2552 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528977 2552 flags.go:64] FLAG: --enable-server="true" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528980 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528985 2552 flags.go:64] FLAG: --event-burst="100" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528988 2552 flags.go:64] FLAG: --event-qps="50" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528990 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528993 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.528998 2552 flags.go:64] FLAG: --eviction-hard="" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529001 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 14:22:45.825301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529004 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529007 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529010 2552 flags.go:64] FLAG: --eviction-soft="" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529013 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529015 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529018 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529021 2552 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529024 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529027 2552 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529029 2552 flags.go:64] FLAG: --feature-gates="" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529033 2552 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529036 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529039 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529042 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529045 2552 flags.go:64] FLAG: --healthz-port="10248" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529048 2552 flags.go:64] FLAG: --help="false" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529051 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-141-213.ec2.internal" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529054 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529058 2552 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529061 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529064 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529068 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529070 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 14:22:45.826197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529073 2552 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529076 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529079 2552 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529082 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529085 2552 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529088 2552 flags.go:64] FLAG: --kube-reserved="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529090 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529095 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529097 2552 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529100 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529103 2552 flags.go:64] FLAG: --lock-file="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529106 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529108 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529111 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529117 2552 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529119 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529122 2552 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529125 2552 flags.go:64] FLAG: --logging-format="text" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529127 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529130 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529133 2552 flags.go:64] FLAG: --manifest-url="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529136 2552 flags.go:64] FLAG: --manifest-url-header="" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529140 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529143 2552 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529147 2552 flags.go:64] FLAG: --max-pods="110" Jul 06 14:22:45.827029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529150 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529153 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529157 2552 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529160 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529163 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529165 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529168 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529176 2552 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529179 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529182 2552 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529185 2552 flags.go:64] FLAG: --pod-cidr="" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529188 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529194 2552 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529197 2552 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529202 2552 flags.go:64] FLAG: --pods-per-core="0" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529204 2552 flags.go:64] FLAG: --port="10250" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529207 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529210 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a75d4baea42d9467" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529214 2552 flags.go:64] FLAG: --qos-reserved="" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529217 2552 flags.go:64] FLAG: --read-only-port="10255" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529220 2552 flags.go:64] FLAG: --register-node="true" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529222 2552 flags.go:64] FLAG: --register-with-taints="" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529226 2552 flags.go:64] FLAG: --registry-burst="10" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529229 2552 flags.go:64] FLAG: --registry-qps="5" Jul 06 14:22:45.827855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529232 2552 flags.go:64] FLAG: --reserved-cpus="" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529235 2552 flags.go:64] FLAG: --reserved-memory="" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529238 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529241 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529244 2552 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529247 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529250 2552 flags.go:64] FLAG: --runonce="false" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529252 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529255 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529258 2552 flags.go:64] FLAG: --seccomp-default="false" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529261 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529266 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529269 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529272 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529275 2552 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529278 2552 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529281 2552 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529284 2552 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529287 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529290 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529294 2552 flags.go:64] FLAG: --system-cgroups="" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529296 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529303 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529306 2552 flags.go:64] FLAG: --tls-cert-file="" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529309 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 14:22:45.828894 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529313 2552 flags.go:64] FLAG: --tls-min-version="" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529316 2552 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529333 2552 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529336 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529339 2552 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529342 2552 flags.go:64] FLAG: --v="2" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529346 2552 flags.go:64] FLAG: --version="false" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529350 2552 flags.go:64] FLAG: --vmodule="" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529353 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.529356 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529474 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529477 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529480 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529483 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529485 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529488 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529491 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529493 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529498 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529500 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529503 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529505 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.829705 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529508 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529511 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529513 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529515 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529518 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529521 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529523 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529527 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529530 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529533 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529535 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529537 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529540 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529542 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529545 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529547 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529550 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529552 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529555 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529558 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.830368 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529560 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529563 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529565 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529567 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529570 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529573 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529576 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529578 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529582 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529585 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529587 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529590 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529592 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529595 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529597 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529600 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529602 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529605 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529608 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529612 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.831012 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529615 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529617 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529620 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529622 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529624 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529627 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529629 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529632 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529634 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529637 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529641 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529645 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529649 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529652 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529655 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529657 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529660 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529662 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.831643 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529665 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529669 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529674 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529677 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529680 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529683 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529686 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529689 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529691 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529694 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529697 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529700 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529702 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529707 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529710 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529712 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529714 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529717 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529719 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529722 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:45.832214 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529724 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529727 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529729 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529732 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529734 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529737 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529740 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529742 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.529744 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.530684 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.537628 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.537647 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.537664 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.537670 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537731 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.832854 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537736 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537739 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537743 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537746 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537749 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537752 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537754 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537757 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537760 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537762 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537765 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537767 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537770 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537773 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537776 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537779 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537782 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537784 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537787 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537790 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:45.833482 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537792 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537795 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537798 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537802 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537805 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537808 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537811 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537813 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537823 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537830 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537833 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537835 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537838 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537841 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537843 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537846 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537848 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537851 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537854 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.834034 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537894 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537898 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537900 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537903 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537907 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537909 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537925 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537930 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.537998 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538001 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538069 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538127 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538130 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538133 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538135 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538138 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538346 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538349 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538534 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538547 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:45.834686 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538550 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538553 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538757 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538764 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538957 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538960 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538963 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538966 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.538969 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539048 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539128 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539131 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539145 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539148 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539150 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539153 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539156 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539159 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539161 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539165 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:45.835221 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539168 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539170 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539173 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539176 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539178 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539181 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539183 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539186 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539189 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539191 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539194 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539196 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539200 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539204 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539207 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539215 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539218 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539222 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.835843 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.539224 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.539231 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540370 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540387 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540390 2552 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540394 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540396 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540399 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540402 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540404 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540407 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540411 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540414 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540417 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540421 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:22:45.836347 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540424 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540427 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540429 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540432 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540435 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540438 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540441 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540443 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540446 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540448 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540451 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540453 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540456 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540458 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540462 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540472 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540474 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540477 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540480 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540482 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:22:45.836821 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540485 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540487 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540490 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540492 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540495 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540498 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540500 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540503 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540506 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540509 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540512 2552 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540516 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540519 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540522 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540525 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540527 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540530 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540532 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540535 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540537 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:22:45.837374 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540539 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540542 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540544 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540547 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540549 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540552 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540554 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540557 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540568 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540571 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540573 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540576 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540579 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540582 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540584 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540586 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540589 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540593 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540595 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:22:46.003765 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540598 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540603 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540606 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540608 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540611 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540613 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540616 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540619 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540621 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540624 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540626 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540628 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540631 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540635 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540638 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540641 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540644 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540646 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540652 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:22:46.006331 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540654 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540657 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540659 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540667 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540670 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540673 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540675 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:45.540678 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.540683 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.540809 2552 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.543593 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.544972 2552 server.go:1019] "Starting client certificate rotation" Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.545073 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:22:46.006949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.545107 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.547731 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.549448 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.562751 2552 log.go:25] "Validated CRI v1 runtime API" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.564754 2552 log.go:25] "Validated CRI v1 image API" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.565613 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.567969 2552 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ba3a7d42-0ecc-4ddc-a35b-d18e5eebe5d3:/dev/nvme0n1p4 c2bb8a56-0137-46c7-935d-c4f4c3d77511:/dev/nvme0n1p3] Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.567985 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 14:22:46.008073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.572585 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.575796 2552 manager.go:216] Machine: {Timestamp:2026-07-06 14:22:45.573586643 +0000 UTC m=+0.344608965 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099539 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28afadaccf6ac433b32b81ee8a3f79 SystemUUID:ec28afad-accf-6ac4-33b3-2b81ee8a3f79 BootID:e4be31d9-def8-4126-acb6-bb0f456e2e79 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:8d:a5:47:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:8d:a5:47:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:c7:d5:5c:59:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.576976 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.577107 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.580495 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.580524 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.581373 2552 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.581383 2552 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.581394 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.581992 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.583356 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586518 2552 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586699 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586735 2552 kubelet.go:486] "Attempting to sync node with API server" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586753 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586798 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586811 2552 kubelet.go:398] "Adding apiserver pod source" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.586841 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 14:22:46.211175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.587439 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.587460 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.589789 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.592671 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.592718 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593912 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593932 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593942 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593952 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593964 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.593974 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594295 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594302 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594310 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594317 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594350 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.594358 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.595029 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.595039 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.599936 2552 server.go:1268] "Started kubelet" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.600725 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.601211 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.601272 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.602027 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.602061 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.602056 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.603292 2552 server.go:310] "Adding debug handlers to kubelet server" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.603297 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.605085 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-213.ec2.internal.18bfb9581f91b79e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-213.ec2.internal,UID:ip-10-0-141-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-213.ec2.internal,},FirstTimestamp:2026-07-06 14:22:45.59990979 +0000 UTC m=+0.370932111,LastTimestamp:2026-07-06 14:22:45.59990979 +0000 UTC m=+0.370932111,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-213.ec2.internal,}" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.607301 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.608587 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.608600 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.608780 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609130 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609146 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609182 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609273 2552 reconstruct.go:97] "Volume reconstruction finished" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609279 2552 reconciler.go:29] "Reconciler: start to sync state" Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609910 2552 factory.go:55] Registering systemd factory Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.609926 2552 factory.go:223] Registration of the systemd container factory successfully Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610125 2552 factory.go:153] Registering CRI-O factory Jul 06 14:22:46.212695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610138 2552 factory.go:223] Registration of the crio container factory successfully Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610186 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610204 2552 factory.go:103] Registering Raw factory Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610214 2552 manager.go:1196] Started watching for new ooms in manager Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.610662 2552 manager.go:349] Starting recovery of all containers Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.618808 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.618826 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.621656 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jlbkm" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.622947 2552 manager.go:354] Recovery completed Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.628635 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.628905 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jlbkm" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.631581 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.631605 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.631615 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632111 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632120 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632134 2552 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632137 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632913 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632936 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.632945 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.633232 2552 policy_none.go:49] "None policy: Start" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.633243 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.633253 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.634199 2552 policy_none.go:47] "Start" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.680914 2552 manager.go:337] "Starting Device Plugin manager" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.681118 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681132 2552 server.go:87] "Starting device plugin registration server" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681443 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681458 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681557 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681646 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.681655 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.682236 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.682287 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.746561 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.747777 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.747814 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.747839 2552 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.747877 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.750540 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.782475 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.783581 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.783611 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.783624 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.783651 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.213963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.792891 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.792913 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-213.ec2.internal\": node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.823044 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.848292 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal"] Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.848378 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.849854 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.849891 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.849908 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851022 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851164 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851198 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851916 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851939 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851921 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851973 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851983 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.851949 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.853141 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.853161 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.854266 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.854297 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:45.854307 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.887053 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.892029 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-213.ec2.internal\" not found" node="ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:45.923739 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.011750 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd54470ed1e6a3a6cacd405db1be8dc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"9fd54470ed1e6a3a6cacd405db1be8dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.011777 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.011801 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.024825 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112542 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.215385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112584 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112607 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd54470ed1e6a3a6cacd405db1be8dc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"9fd54470ed1e6a3a6cacd405db1be8dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112636 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112654 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f4e64050314bf7afb8f9308e8868ad1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal\" (UID: \"2f4e64050314bf7afb8f9308e8868ad1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.112675 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd54470ed1e6a3a6cacd405db1be8dc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-213.ec2.internal\" (UID: \"9fd54470ed1e6a3a6cacd405db1be8dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.125627 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.191811 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.216211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.200064 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:46.387804 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.226114 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.387804 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.326569 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.427005 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.496546 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.528012 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.544288 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.544440 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:22:46.577876 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.544467 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:22:46.729041 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.628878 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.729041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.631025 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:17:45 +0000 UTC" deadline="2028-02-15 17:31:34.819010167 +0000 UTC" Jul 06 14:22:46.729041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.631050 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14139h8m48.187963542s" Jul 06 14:22:46.729041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.703643 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:46.732723 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.732403 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.857226 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:46.833495 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-213.ec2.internal\" not found" Jul 06 14:22:46.857226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.846431 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.913593 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:46.917505 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4e64050314bf7afb8f9308e8868ad1.slice/crio-029ff540d7fd775482401357ff1f72e7865714366c34eb16c774a938e33fec9f WatchSource:0}: Error finding container 029ff540d7fd775482401357ff1f72e7865714366c34eb16c774a938e33fec9f: Status 404 returned error can't find the container with id 029ff540d7fd775482401357ff1f72e7865714366c34eb16c774a938e33fec9f Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:46.917932 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd54470ed1e6a3a6cacd405db1be8dc.slice/crio-3acec414cb30b76bf45afafb1c5a9616fbde5722507afc7dfffaaf777051cf0a WatchSource:0}: Error finding container 3acec414cb30b76bf45afafb1c5a9616fbde5722507afc7dfffaaf777051cf0a: Status 404 returned error can't find the container with id 3acec414cb30b76bf45afafb1c5a9616fbde5722507afc7dfffaaf777051cf0a Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.922023 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.927430 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.928331 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" Jul 06 14:22:47.033036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:46.936269 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.589630 2552 apiserver.go:52] "Watching apiserver" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.598162 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.599073 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7q979","kube-system/konnectivity-agent-fcbgm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k","openshift-cluster-node-tuning-operator/tuned-p75pj","openshift-dns/node-resolver-kwqqr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal","openshift-multus/multus-additional-cni-plugins-8wfl4","openshift-multus/multus-fvnmr","kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal","openshift-image-registry/node-ca-dx8bm","openshift-multus/network-metrics-daemon-gvwjb","openshift-network-diagnostics/network-check-target-z8xcc","openshift-network-operator/iptables-alerter-2lvgw","openshift-ovn-kubernetes/ovnkube-node-2ftg5"] Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.600622 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.602068 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.602138 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.602539 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.602657 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.602826 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.602858 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.603023 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4gfqs\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.603066 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.603727 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.604621 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.605105 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.605468 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5h7tq\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.605708 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.605929 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.606153 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.607022 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.607094 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.607149 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ss4mf\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.607614 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.607672 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.608526 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.609612 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.609997 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.610215 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.610303 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f7qzc\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.610401 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2lvfv\"" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.612513 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.644727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.613391 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.614367 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xfqch\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.614416 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.614427 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.614721 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.615033 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.616566 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.616774 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.616933 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c5dqx\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.617096 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.617338 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.617393 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.618434 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619247 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-bin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619275 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48ngb\" (UniqueName: \"kubernetes.io/projected/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-kube-api-access-48ngb\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619300 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f39976-8fee-49d6-9e1e-6d27d2186e6e-agent-certs\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619341 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6mzb\" (UniqueName: \"kubernetes.io/projected/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kube-api-access-z6mzb\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619364 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-conf\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619384 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-host\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619404 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9vg8\" (UniqueName: \"kubernetes.io/projected/2c6588d3-6b5b-4fdc-b416-73267b1806d0-kube-api-access-f9vg8\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619424 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cnibin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619446 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-multus\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.647907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619466 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-os-release\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619486 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619506 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f39976-8fee-49d6-9e1e-6d27d2186e6e-konnectivity-ca\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619526 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619548 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-device-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619561 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619569 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619591 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-sys-fs\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619651 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-system-cni-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619674 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619695 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-conf-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619715 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/338fb009-c0a9-46a1-9a89-69a41bc7959f-iptables-alerter-script\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619741 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-registration-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619774 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-var-lib-kubelet\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619795 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-socket-dir-parent\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.649405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619817 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-kubelet\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619838 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619868 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmrwg\" (UniqueName: \"kubernetes.io/projected/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-kube-api-access-lmrwg\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619889 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lkpp\" (UniqueName: \"kubernetes.io/projected/338fb009-c0a9-46a1-9a89-69a41bc7959f-kube-api-access-5lkpp\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619910 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-sys\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619931 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-lib-modules\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619965 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.619986 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cni-binary-copy\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620012 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-k8s-cni-cncf-io\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620035 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620062 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-etc-kubernetes\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620083 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysconfig\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620111 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-kubernetes\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620132 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-os-release\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620153 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-socket-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620173 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-run\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.651696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-system-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620226 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-netns\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620258 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-daemon-config\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620281 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-multus-certs\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620302 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/338fb009-c0a9-46a1-9a89-69a41bc7959f-host-slash\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620344 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620380 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620404 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-tuned\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn87j\" (UniqueName: \"kubernetes.io/projected/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-kube-api-access-nn87j\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.620998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-hostroot\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.621063 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cnibin\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.621099 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-modprobe-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.621126 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-systemd\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.621183 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-tmp\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623461 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623520 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623726 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623731 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623761 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623800 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623834 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gdv6c\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.623947 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 14:22:47.653030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.624010 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4666g\"" Jul 06 14:22:47.655618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.631274 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 14:22:47.655618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.632224 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:22:47.655618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.643107 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcgd7" Jul 06 14:22:47.655618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.651288 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcgd7" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.710028 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721786 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-env-overrides\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721829 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-socket-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721860 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-run\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721883 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-system-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721906 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-netns\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721928 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-daemon-config\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721958 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-multus-certs\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721995 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-socket-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722044 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-multus-certs\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722047 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-netns\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.721997 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-hosts-file\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722065 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-run\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/338fb009-c0a9-46a1-9a89-69a41bc7959f-host-slash\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722122 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722144 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722167 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-tuned\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.745615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722189 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722194 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn87j\" (UniqueName: \"kubernetes.io/projected/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-kube-api-access-nn87j\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722227 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722252 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-hostroot\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722269 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722279 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cnibin\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722309 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-modprobe-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722316 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-hostroot\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722357 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-systemd\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-tmp\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722402 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-bin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722424 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48ngb\" (UniqueName: \"kubernetes.io/projected/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-kube-api-access-48ngb\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722169 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/338fb009-c0a9-46a1-9a89-69a41bc7959f-host-slash\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722447 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f39976-8fee-49d6-9e1e-6d27d2186e6e-agent-certs\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722472 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-kubelet\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722497 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6mzb\" (UniqueName: \"kubernetes.io/projected/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kube-api-access-z6mzb\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722502 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-systemd\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.747781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722522 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-conf\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722542 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cnibin\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722545 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-host\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722560 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-daemon-config\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722578 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9vg8\" (UniqueName: \"kubernetes.io/projected/2c6588d3-6b5b-4fdc-b416-73267b1806d0-kube-api-access-f9vg8\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722587 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-host\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722606 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cnibin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722616 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722630 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-multus\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722656 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-system-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722657 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-822r4\" (UniqueName: \"kubernetes.io/projected/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-kube-api-access-822r4\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722705 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cnibin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722745 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-multus\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.722760 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-cni-bin\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723219 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-os-release\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723284 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723315 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-node-log\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723342 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-modprobe-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.748477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723421 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-os-release\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723499 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-conf\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.723505 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723352 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-netd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723548 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f39976-8fee-49d6-9e1e-6d27d2186e6e-konnectivity-ca\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.723576 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:48.223553729 +0000 UTC m=+2.994576066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723607 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723634 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-kubelet-config\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723659 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723685 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-device-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723733 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-tmp-dir\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723755 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-var-lib-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723780 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-sys-fs\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723805 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723828 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-system-cni-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.751680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723829 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-device-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723853 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-bin\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723907 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-host\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723936 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723944 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-sys-fs\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723974 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-conf-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.723995 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/11f39976-8fee-49d6-9e1e-6d27d2186e6e-konnectivity-ca\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724001 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/338fb009-c0a9-46a1-9a89-69a41bc7959f-iptables-alerter-script\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724044 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-etc-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-ovn\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724094 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724120 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sql8z\" (UniqueName: \"kubernetes.io/projected/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-kube-api-access-sql8z\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724143 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-serviceca\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724202 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-system-cni-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724239 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-registration-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-var-lib-kubelet\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724287 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-socket-dir-parent\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.752335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724312 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-systemd-units\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724312 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724356 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-kubelet\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724457 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysctl-d\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724498 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0fa4d6a-69f1-4208-bc6f-9e48da342570-registration-dir\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724531 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-conf-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724571 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-var-lib-kubelet\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724569 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/338fb009-c0a9-46a1-9a89-69a41bc7959f-iptables-alerter-script\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724619 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-socket-dir-parent\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724619 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-var-lib-kubelet\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724658 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724706 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmrwg\" (UniqueName: \"kubernetes.io/projected/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-kube-api-access-lmrwg\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724855 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lkpp\" (UniqueName: \"kubernetes.io/projected/338fb009-c0a9-46a1-9a89-69a41bc7959f-kube-api-access-5lkpp\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724880 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724891 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wd4\" (UniqueName: \"kubernetes.io/projected/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-kube-api-access-s2wd4\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.724987 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-slash\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725014 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-sys\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.755305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725064 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-lib-modules\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725130 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-sys\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725131 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725187 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cni-binary-copy\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725195 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-lib-modules\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725237 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-multus-cni-dir\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725237 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-k8s-cni-cncf-io\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725294 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-host-run-k8s-cni-cncf-io\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725315 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725346 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725386 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-log-socket\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725416 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-script-lib\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725441 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-etc-kubernetes\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725489 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-netns\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725504 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-etc-kubernetes\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725548 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-systemd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725576 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysconfig\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.756195 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725614 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-kubernetes\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725623 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-sysconfig\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725647 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-cni-binary-copy\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725654 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-os-release\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725672 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-kubernetes\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725729 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-os-release\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725750 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-dbus\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725787 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725906 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.725933 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-config\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.726761 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-tmp\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.726797 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2c6588d3-6b5b-4fdc-b416-73267b1806d0-etc-tuned\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.728419 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/11f39976-8fee-49d6-9e1e-6d27d2186e6e-agent-certs\") pod \"konnectivity-agent-fcbgm\" (UID: \"11f39976-8fee-49d6-9e1e-6d27d2186e6e\") " pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.737043 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.737068 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.737080 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:47.758264 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.737176 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:22:48.237137948 +0000 UTC m=+3.008160262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.739733 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6mzb\" (UniqueName: \"kubernetes.io/projected/c0fa4d6a-69f1-4208-bc6f-9e48da342570-kube-api-access-z6mzb\") pod \"aws-ebs-csi-driver-node-bzp6k\" (UID: \"c0fa4d6a-69f1-4208-bc6f-9e48da342570\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.741751 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9vg8\" (UniqueName: \"kubernetes.io/projected/2c6588d3-6b5b-4fdc-b416-73267b1806d0-kube-api-access-f9vg8\") pod \"tuned-p75pj\" (UID: \"2c6588d3-6b5b-4fdc-b416-73267b1806d0\") " pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.741839 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmrwg\" (UniqueName: \"kubernetes.io/projected/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-kube-api-access-lmrwg\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.742186 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn87j\" (UniqueName: \"kubernetes.io/projected/d1e77941-bbf8-4e5a-b118-61d68b12bdbb-kube-api-access-nn87j\") pod \"multus-additional-cni-plugins-8wfl4\" (UID: \"d1e77941-bbf8-4e5a-b118-61d68b12bdbb\") " pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.743731 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48ngb\" (UniqueName: \"kubernetes.io/projected/fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8-kube-api-access-48ngb\") pod \"multus-fvnmr\" (UID: \"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8\") " pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.743781 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lkpp\" (UniqueName: \"kubernetes.io/projected/338fb009-c0a9-46a1-9a89-69a41bc7959f-kube-api-access-5lkpp\") pod \"iptables-alerter-2lvgw\" (UID: \"338fb009-c0a9-46a1-9a89-69a41bc7959f\") " pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.754354 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"2f4e64050314bf7afb8f9308e8868ad1","Type":"ContainerStarted","Data":"029ff540d7fd775482401357ff1f72e7865714366c34eb16c774a938e33fec9f"} Jul 06 14:22:47.758870 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.755384 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" event={"ID":"9fd54470ed1e6a3a6cacd405db1be8dc","Type":"ContainerStarted","Data":"3acec414cb30b76bf45afafb1c5a9616fbde5722507afc7dfffaaf777051cf0a"} Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826514 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-dbus\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826557 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826588 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826640 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-config\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826668 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-env-overrides\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826699 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-hosts-file\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826771 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-hosts-file\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826804 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-dbus\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826817 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.826820 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:47.826885 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:48.326864055 +0000 UTC m=+3.097886378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826898 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826901 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-kubelet\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826929 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-kubelet\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826936 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-822r4\" (UniqueName: \"kubernetes.io/projected/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-kube-api-access-822r4\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.854110 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.826975 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-node-log\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.827059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-netd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.827165 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-node-log\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.827496 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-env-overrides\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.827193 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-netd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.827263 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-config\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.828130 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-kubelet-config\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.828767 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.828990 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-tmp-dir\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829223 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-var-lib-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829264 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-bin\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-host\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829313 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-etc-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829383 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-ovn\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829407 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829431 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sql8z\" (UniqueName: \"kubernetes.io/projected/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-kube-api-access-sql8z\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829454 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-serviceca\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829484 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-systemd-units\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.854985 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829512 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wd4\" (UniqueName: \"kubernetes.io/projected/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-kube-api-access-s2wd4\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829533 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-slash\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829563 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-log-socket\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829586 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-script-lib\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-netns\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829632 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-systemd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.829701 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-systemd\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.828429 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b1f8d040-e6ef-4596-a93e-b22ff5976566-kubelet-config\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830442 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-tmp-dir\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830494 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-var-lib-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830534 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-cni-bin\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830568 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-host\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830602 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-etc-openvswitch\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830636 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-run-ovn\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830670 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830705 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-slash\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830737 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-systemd-units\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830794 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-serviceca\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.858373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830859 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-log-socket\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.830922 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-host-run-netns\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.831379 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovnkube-script-lib\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.832312 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.835218 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-822r4\" (UniqueName: \"kubernetes.io/projected/c35f45ce-4d40-4bb9-b18e-e6eb9917a1db-kube-api-access-822r4\") pod \"node-ca-dx8bm\" (UID: \"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db\") " pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.837840 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wd4\" (UniqueName: \"kubernetes.io/projected/3cb3e28a-fffd-47d3-b379-e3bed6b8b03f-kube-api-access-s2wd4\") pod \"node-resolver-kwqqr\" (UID: \"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f\") " pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.859659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.838721 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sql8z\" (UniqueName: \"kubernetes.io/projected/23e7f036-b2b0-499a-a66e-fa1c90fc05dc-kube-api-access-sql8z\") pod \"ovnkube-node-2ftg5\" (UID: \"23e7f036-b2b0-499a-a66e-fa1c90fc05dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.913343 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.924408 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.928200 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.936825 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p75pj" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.948847 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvnmr" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.952000 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.964779 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2lvgw" Jul 06 14:22:47.979855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.973973 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dx8bm" Jul 06 14:22:47.986137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.982656 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kwqqr" Jul 06 14:22:47.999224 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:47.999194 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:22:48.238370 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.238273 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:48.238516 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238428 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:48.238516 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.238444 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:48.238516 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238498 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:49.238480009 +0000 UTC m=+4.009502316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:48.238679 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238577 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:22:48.238679 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238600 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:22:48.238679 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238613 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:48.238679 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.238666 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:22:49.238651495 +0000 UTC m=+4.009673805 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:48.338882 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.338846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:48.339031 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.339011 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:48.339091 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:48.339080 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:49.339063701 +0000 UTC m=+4.110086013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:48.654105 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.654005 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:17:47 +0000 UTC" deadline="2028-02-24 09:10:11.567672708 +0000 UTC" Jul 06 14:22:48.654105 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.654049 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14346h47m22.913628749s" Jul 06 14:22:48.916784 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.916760 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6588d3_6b5b_4fdc_b416_73267b1806d0.slice/crio-a59f3d56682f6fed204bcb7b8e6e73dbd78812f7703a8f2c6264edbfb0a3a8f9 WatchSource:0}: Error finding container a59f3d56682f6fed204bcb7b8e6e73dbd78812f7703a8f2c6264edbfb0a3a8f9: Status 404 returned error can't find the container with id a59f3d56682f6fed204bcb7b8e6e73dbd78812f7703a8f2c6264edbfb0a3a8f9 Jul 06 14:22:48.918715 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.918692 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f39976_8fee_49d6_9e1e_6d27d2186e6e.slice/crio-0175d777641fc86c92eef8b2fce2c1d5999cff2101b9ba7f7bdbbcfbe4d27940 WatchSource:0}: Error finding container 0175d777641fc86c92eef8b2fce2c1d5999cff2101b9ba7f7bdbbcfbe4d27940: Status 404 returned error can't find the container with id 0175d777641fc86c92eef8b2fce2c1d5999cff2101b9ba7f7bdbbcfbe4d27940 Jul 06 14:22:48.921052 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.920999 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e77941_bbf8_4e5a_b118_61d68b12bdbb.slice/crio-f43d5b7f43ebb7089ae18dac575c41514bafd6627a5ae1ff6959bc652b2234aa WatchSource:0}: Error finding container f43d5b7f43ebb7089ae18dac575c41514bafd6627a5ae1ff6959bc652b2234aa: Status 404 returned error can't find the container with id f43d5b7f43ebb7089ae18dac575c41514bafd6627a5ae1ff6959bc652b2234aa Jul 06 14:22:48.924068 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.923982 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4ce6ed_ab49_4459_8d1d_f7e72fbb40c8.slice/crio-56e6fbf329af3b7df02b5c64ca6c6bbf98dcc52baf49824a25f8ab63adc1163b WatchSource:0}: Error finding container 56e6fbf329af3b7df02b5c64ca6c6bbf98dcc52baf49824a25f8ab63adc1163b: Status 404 returned error can't find the container with id 56e6fbf329af3b7df02b5c64ca6c6bbf98dcc52baf49824a25f8ab63adc1163b Jul 06 14:22:48.924986 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.924967 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb3e28a_fffd_47d3_b379_e3bed6b8b03f.slice/crio-63dafa3a32672e0f152be41d33b66e5f0cfc076f7b964dad1c2f62faa9ad7da1 WatchSource:0}: Error finding container 63dafa3a32672e0f152be41d33b66e5f0cfc076f7b964dad1c2f62faa9ad7da1: Status 404 returned error can't find the container with id 63dafa3a32672e0f152be41d33b66e5f0cfc076f7b964dad1c2f62faa9ad7da1 Jul 06 14:22:48.925704 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.925634 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e7f036_b2b0_499a_a66e_fa1c90fc05dc.slice/crio-c1cba4a018532e6466c6b8796881b57f391470cb949a1c224cc0bb218e6750cd WatchSource:0}: Error finding container c1cba4a018532e6466c6b8796881b57f391470cb949a1c224cc0bb218e6750cd: Status 404 returned error can't find the container with id c1cba4a018532e6466c6b8796881b57f391470cb949a1c224cc0bb218e6750cd Jul 06 14:22:48.926923 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.926901 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35f45ce_4d40_4bb9_b18e_e6eb9917a1db.slice/crio-bf63dc4a7b62a8ae1a70a3179bcf28e471f8563a928a06e61251996881786423 WatchSource:0}: Error finding container bf63dc4a7b62a8ae1a70a3179bcf28e471f8563a928a06e61251996881786423: Status 404 returned error can't find the container with id bf63dc4a7b62a8ae1a70a3179bcf28e471f8563a928a06e61251996881786423 Jul 06 14:22:48.928217 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:22:48.928172 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338fb009_c0a9_46a1_9a89_69a41bc7959f.slice/crio-56f8898ca6aa87910f859819222890d020488a8fb62c2bcb1ad6bf87fa31e3b0 WatchSource:0}: Error finding container 56f8898ca6aa87910f859819222890d020488a8fb62c2bcb1ad6bf87fa31e3b0: Status 404 returned error can't find the container with id 56f8898ca6aa87910f859819222890d020488a8fb62c2bcb1ad6bf87fa31e3b0 Jul 06 14:22:48.956200 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:48.956175 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:22:49.245270 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.245079 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:49.245425 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.245288 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:49.245425 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245233 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:49.245425 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245395 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:51.245379743 +0000 UTC m=+6.016402054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:49.245425 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245408 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:22:49.245425 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245427 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:22:49.245618 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245440 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:49.245618 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.245485 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:22:51.245470275 +0000 UTC m=+6.016492604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:49.345765 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.345736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:49.345908 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.345889 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:49.345964 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.345954 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:51.345936987 +0000 UTC m=+6.116959305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:49.654511 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.654424 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:17:47 +0000 UTC" deadline="2028-02-27 18:10:37.512401578 +0000 UTC" Jul 06 14:22:49.654511 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.654468 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14427h47m47.857938632s" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.748431 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.748553 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.748625 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.748697 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.748757 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:49.751031 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:49.748837 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:49.778080 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.777981 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" event={"ID":"c0fa4d6a-69f1-4208-bc6f-9e48da342570","Type":"ContainerStarted","Data":"ef321469980d267c40bc947af9abb9b56edacb652edc27f3ac484c9cec096e54"} Jul 06 14:22:49.783657 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.783626 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dx8bm" event={"ID":"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db","Type":"ContainerStarted","Data":"bf63dc4a7b62a8ae1a70a3179bcf28e471f8563a928a06e61251996881786423"} Jul 06 14:22:49.788385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.788355 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"c1cba4a018532e6466c6b8796881b57f391470cb949a1c224cc0bb218e6750cd"} Jul 06 14:22:49.793943 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.793914 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kwqqr" event={"ID":"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f","Type":"ContainerStarted","Data":"63dafa3a32672e0f152be41d33b66e5f0cfc076f7b964dad1c2f62faa9ad7da1"} Jul 06 14:22:49.800284 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.800255 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvnmr" event={"ID":"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8","Type":"ContainerStarted","Data":"56e6fbf329af3b7df02b5c64ca6c6bbf98dcc52baf49824a25f8ab63adc1163b"} Jul 06 14:22:49.807718 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.807665 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerStarted","Data":"f43d5b7f43ebb7089ae18dac575c41514bafd6627a5ae1ff6959bc652b2234aa"} Jul 06 14:22:49.815795 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.815763 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p75pj" event={"ID":"2c6588d3-6b5b-4fdc-b416-73267b1806d0","Type":"ContainerStarted","Data":"a59f3d56682f6fed204bcb7b8e6e73dbd78812f7703a8f2c6264edbfb0a3a8f9"} Jul 06 14:22:49.823635 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.823604 2552 generic.go:356] "Generic (PLEG): container finished" podID="2f4e64050314bf7afb8f9308e8868ad1" containerID="6db39905a70380c5360b9de3578ffdd331bdc56d4abda54b936328aaed812b66" exitCode=0 Jul 06 14:22:49.823754 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.823688 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"2f4e64050314bf7afb8f9308e8868ad1","Type":"ContainerDied","Data":"6db39905a70380c5360b9de3578ffdd331bdc56d4abda54b936328aaed812b66"} Jul 06 14:22:49.828202 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.828178 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" event={"ID":"9fd54470ed1e6a3a6cacd405db1be8dc","Type":"ContainerStarted","Data":"0be5fa8c3ca784f5fbc46fedbae86a22aa6900c443f6335c9ea28c23924189f7"} Jul 06 14:22:49.837116 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.837090 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2lvgw" event={"ID":"338fb009-c0a9-46a1-9a89-69a41bc7959f","Type":"ContainerStarted","Data":"56f8898ca6aa87910f859819222890d020488a8fb62c2bcb1ad6bf87fa31e3b0"} Jul 06 14:22:49.839710 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:49.839647 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcbgm" event={"ID":"11f39976-8fee-49d6-9e1e-6d27d2186e6e","Type":"ContainerStarted","Data":"0175d777641fc86c92eef8b2fce2c1d5999cff2101b9ba7f7bdbbcfbe4d27940"} Jul 06 14:22:50.895108 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:50.895072 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" event={"ID":"2f4e64050314bf7afb8f9308e8868ad1","Type":"ContainerStarted","Data":"5f3285c71850e323f01126805bf4372accbfac4149d0bcc865ffeef4e4104980"} Jul 06 14:22:50.908423 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:50.908354 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-213.ec2.internal" podStartSLOduration=4.908316443 podStartE2EDuration="4.908316443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:49.85295999 +0000 UTC m=+4.623982319" watchObservedRunningTime="2026-07-06 14:22:50.908316443 +0000 UTC m=+5.679338773" Jul 06 14:22:51.262213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.262126 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:51.262213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.262184 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:51.262438 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262345 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:22:51.262438 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262366 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:22:51.262438 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262379 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:51.262438 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262437 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:22:55.26241881 +0000 UTC m=+10.033441135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:51.262865 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262845 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:51.262976 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.262903 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:55.262885039 +0000 UTC m=+10.033907352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:51.362788 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.362744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:51.362979 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.362944 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:51.363039 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.363008 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:22:55.362989868 +0000 UTC m=+10.134012180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.748579 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.748704 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.748839 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:51.748839 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.748955 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:51.749181 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:51.749041 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:53.749074 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:53.749042 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:53.749585 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:53.749171 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:53.749585 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:53.749167 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:53.749585 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:53.749277 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:53.749585 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:53.749338 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:53.749585 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:53.749412 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:55.297192 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.297141 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.297209 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297409 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297432 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297446 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297506 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:23:03.297486951 +0000 UTC m=+18.068509282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:22:55.297701 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297653 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:55.298025 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.297712 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:03.297695598 +0000 UTC m=+18.068717911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:22:55.398179 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.398141 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:55.398359 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.398290 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:55.398426 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.398368 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:03.398348914 +0000 UTC m=+18.169371243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:22:55.752764 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.752673 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:55.752911 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.752792 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:55.753447 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.753338 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:55.753591 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.753453 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:55.753591 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:55.753513 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:55.753591 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:55.753580 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:57.748925 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:57.748892 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:57.749405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:57.748892 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:57.749405 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:57.749017 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:57.749405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:57.749036 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:57.749405 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:57.749094 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:57.749405 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:57.749158 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:59.749720 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.749107 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:22:59.749720 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:59.749462 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:22:59.749720 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.749619 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:22:59.750224 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:59.749736 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:22:59.750224 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.749803 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:22:59.750224 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:22:59.749895 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:22:59.917299 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.917070 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kwqqr" event={"ID":"3cb3e28a-fffd-47d3-b379-e3bed6b8b03f","Type":"ContainerStarted","Data":"314494ec70bf64b53338a319439dd6c3db36ee82937cc06b951829d9b6847afd"} Jul 06 14:22:59.919148 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.919118 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="cfa60657eb57faaf311db323c35381f71f0fa90becafad98904f2217a4d6a42e" exitCode=0 Jul 06 14:22:59.919288 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.919209 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"cfa60657eb57faaf311db323c35381f71f0fa90becafad98904f2217a4d6a42e"} Jul 06 14:22:59.921379 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.921352 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p75pj" event={"ID":"2c6588d3-6b5b-4fdc-b416-73267b1806d0","Type":"ContainerStarted","Data":"6ac92908291f4885ecfab33d77ad9b450e663109ad6af5698d3582b712199c71"} Jul 06 14:22:59.923024 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.922959 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcbgm" event={"ID":"11f39976-8fee-49d6-9e1e-6d27d2186e6e","Type":"ContainerStarted","Data":"248eb5d62e8981b6c7f350da8d9de50a114281ab244fa8faece78925b8a64b06"} Jul 06 14:22:59.925035 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.925013 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" event={"ID":"c0fa4d6a-69f1-4208-bc6f-9e48da342570","Type":"ContainerStarted","Data":"dd4ea270a9e75ce05a4a3110f91d13803e31c2be688a11b880ddef1660304480"} Jul 06 14:22:59.926681 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.926657 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dx8bm" event={"ID":"c35f45ce-4d40-4bb9-b18e-e6eb9917a1db","Type":"ContainerStarted","Data":"165913a595f0725d201ceed570f37de8f94b5083bb2e91b68af8ce006fddf4e9"} Jul 06 14:22:59.933026 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.932967 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kwqqr" podStartSLOduration=3.729324408 podStartE2EDuration="13.932949681s" totalImagesPullingTime="10.203625273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:59.932531508 +0000 UTC m=+14.703553839" watchObservedRunningTime="2026-07-06 14:22:59.932949681 +0000 UTC m=+14.703972012" Jul 06 14:22:59.933151 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.933062 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-213.ec2.internal" podStartSLOduration=13.933056503 podStartE2EDuration="13.933056503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:50.908134391 +0000 UTC m=+5.679156723" watchObservedRunningTime="2026-07-06 14:22:59.933056503 +0000 UTC m=+14.704078834" Jul 06 14:22:59.948374 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.948315 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dx8bm" podStartSLOduration=3.7772342979999998 podStartE2EDuration="13.948299406s" totalImagesPullingTime="10.171065108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:59.948158892 +0000 UTC m=+14.719181225" watchObservedRunningTime="2026-07-06 14:22:59.948299406 +0000 UTC m=+14.719321736" Jul 06 14:22:59.996683 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:22:59.996621 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fcbgm" podStartSLOduration=4.79357294 podStartE2EDuration="14.996597859s" totalImagesPullingTime="10.203024919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:59.995880568 +0000 UTC m=+14.766902910" watchObservedRunningTime="2026-07-06 14:22:59.996597859 +0000 UTC m=+14.767620190" Jul 06 14:23:00.014742 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:00.014673 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p75pj" podStartSLOduration=4.396199566 podStartE2EDuration="15.014647482s" totalImagesPullingTime="10.618447916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:00.013953036 +0000 UTC m=+14.784975367" watchObservedRunningTime="2026-07-06 14:23:00.014647482 +0000 UTC m=+14.785669875" Jul 06 14:23:00.071688 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:00.071570 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:23:00.072249 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:00.072226 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:23:00.931177 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:00.931138 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2lvgw" event={"ID":"338fb009-c0a9-46a1-9a89-69a41bc7959f","Type":"ContainerStarted","Data":"e4576bf95b37740634d9adf9d0b34bc6cb69752ac6dc0d2d57a6b265e6324c0f"} Jul 06 14:23:00.949511 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:00.949449 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2lvgw" podStartSLOduration=5.75599472 podStartE2EDuration="15.949431799s" totalImagesPullingTime="10.193437079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:00.949258993 +0000 UTC m=+15.720281324" watchObservedRunningTime="2026-07-06 14:23:00.949431799 +0000 UTC m=+15.720454129" Jul 06 14:23:01.749081 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:01.749051 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:01.749249 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:01.749051 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:01.749249 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:01.749174 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:01.749413 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:01.749259 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:01.749413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:01.749051 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:01.749413 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:01.749356 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:01.933470 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:01.933440 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 14:23:03.355780 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.355744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.355793 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355872 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355930 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355948 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355950 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:19.355934639 +0000 UTC m=+34.126956951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355958 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:23:03.356384 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.355996 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:23:19.355984405 +0000 UTC m=+34.127006717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:23:03.456207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.456175 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:03.456393 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.456308 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:23:03.456393 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.456391 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:19.456370897 +0000 UTC m=+34.227393208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:23:03.749038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.748942 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:03.749214 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.748955 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:03.749214 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.749088 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:03.749376 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.749204 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:03.749376 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:03.748942 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:03.749516 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:03.749367 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:05.749369 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.749316 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:05.749369 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.749370 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:05.749915 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.749443 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:05.749915 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:05.749476 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:05.749915 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:05.749519 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:05.749915 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:05.749628 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:05.796341 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.796184 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:23:05.796341 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.796345 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 14:23:05.796953 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:05.796928 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fcbgm" Jul 06 14:23:07.748594 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:07.748559 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:07.749083 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:07.748559 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:07.749083 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:07.748686 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:07.749083 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:07.748559 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:07.749083 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:07.748819 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:07.749083 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:07.748901 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:08.868435 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:08.868408 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 14:23:08.949240 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:08.949204 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" event={"ID":"c0fa4d6a-69f1-4208-bc6f-9e48da342570","Type":"ContainerStarted","Data":"ba939bea2a5ad70e502e9c8a69d157757e7d1f99bf24d6c197d500019bc4ab91"} Jul 06 14:23:08.950913 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:08.950877 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"7cb5fc50c4f70f1df9d200dd00ebda60b833634585ba511dbdc8e1688dc7d96a"} Jul 06 14:23:08.952636 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:08.952584 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvnmr" event={"ID":"fa4ce6ed-ab49-4459-8d1d-f7e72fbb40c8","Type":"ContainerStarted","Data":"0cc45e1767114fb17db8139d662fece58efe8769cfbfa079b6ab4769aaf52741"} Jul 06 14:23:08.973644 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:08.973587 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fvnmr" podStartSLOduration=4.133914851 podStartE2EDuration="23.973567073s" totalImagesPullingTime="19.839652222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:08.973179897 +0000 UTC m=+23.744202302" watchObservedRunningTime="2026-07-06 14:23:08.973567073 +0000 UTC m=+23.744589405" Jul 06 14:23:09.697396 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.697263 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:23:08.868430363Z","UUID":"07dca257-d34f-4975-bf4c-e45ddfc4f6af","Handler":null,"Name":"","Endpoint":""} Jul 06 14:23:09.699350 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.699307 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 14:23:09.699508 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.699368 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 14:23:09.748254 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.748221 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:09.748447 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.748270 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:09.748447 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.748220 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:09.748447 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:09.748364 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:09.748591 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:09.748452 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:09.748591 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:09.748548 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:09.957480 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.957397 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:09.957890 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.957865 2552 generic.go:356] "Generic (PLEG): container finished" podID="23e7f036-b2b0-499a-a66e-fa1c90fc05dc" containerID="4a324600b1e3c7125b08d6bdb518c65878e19a6db09cb954de9a8cef828a690a" exitCode=1 Jul 06 14:23:09.957955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.957935 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"198b925860ee6138c03068748ae05caac3300fab035d616daff14ec0c547008c"} Jul 06 14:23:09.958024 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:09.958009 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerDied","Data":"4a324600b1e3c7125b08d6bdb518c65878e19a6db09cb954de9a8cef828a690a"} Jul 06 14:23:10.962590 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:10.962564 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:10.963255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:10.962883 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"5aad28c4e5b3015c5c3989a8e92a4366bc73210c1e782c3c9b844e3438b82216"} Jul 06 14:23:10.963255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:10.962942 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"674f87a81ab9d30d5e251e248eaeeed2e2f877988e5072fc3cbca5281f4a4bf6"} Jul 06 14:23:11.748572 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.748536 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:11.748749 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.748543 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:11.748749 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:11.748639 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:11.748749 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.748659 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:11.748918 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:11.748758 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:11.748918 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:11.748855 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:11.967289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.967264 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:11.967722 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.967624 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"43752395e269af48acf24a2d767db07f7783860457aa8264e8ac6496e1a5c5bd"} Jul 06 14:23:11.969117 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.969091 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="b0af102b00872a9186ea09c008dfde301b70fee76d3bff7152ba56bcf0dfc16f" exitCode=0 Jul 06 14:23:11.969203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:11.969142 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"b0af102b00872a9186ea09c008dfde301b70fee76d3bff7152ba56bcf0dfc16f"} Jul 06 14:23:12.973306 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:12.973271 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" event={"ID":"c0fa4d6a-69f1-4208-bc6f-9e48da342570","Type":"ContainerStarted","Data":"68f0939e6bfbecc9e28288fbadb9ca84df77ab79952c2d97366c6aa60ac01ddb"} Jul 06 14:23:12.992736 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:12.992688 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzp6k" podStartSLOduration=5.677126509 podStartE2EDuration="27.992674176s" totalImagesPullingTime="22.315547667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:12.992278121 +0000 UTC m=+27.763300464" watchObservedRunningTime="2026-07-06 14:23:12.992674176 +0000 UTC m=+27.763696515" Jul 06 14:23:13.748927 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.748896 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:13.749101 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.748899 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:13.749101 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:13.748995 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:13.749101 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.749009 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:13.749223 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:13.749102 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:13.749223 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:13.749186 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:13.980243 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.980221 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:13.980726 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.980706 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"34691367cc955467db686e99b999c9a61183516234a489e108c530f326a6cd36"} Jul 06 14:23:13.982631 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.982608 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="ac37961b7acecbebb3ca06d212569877a8258508f7b5b3fb78d22c2e793b32ad" exitCode=0 Jul 06 14:23:13.982736 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:13.982692 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"ac37961b7acecbebb3ca06d212569877a8258508f7b5b3fb78d22c2e793b32ad"} Jul 06 14:23:15.749379 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.749105 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:15.749379 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.749180 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:15.749379 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:15.749371 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:15.749878 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:15.749467 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:15.749878 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.749204 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:15.749878 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:15.749573 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:15.990023 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.989996 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:15.990420 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.990386 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"4aee268d3d2f87b70937eae74d4bdb90973b57d2e3a831e3611df750662b3b93"} Jul 06 14:23:15.990637 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.990596 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:15.990756 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.990672 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:15.990756 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.990721 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:15.990863 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.990837 2552 scope.go:117] "RemoveContainer" containerID="4a324600b1e3c7125b08d6bdb518c65878e19a6db09cb954de9a8cef828a690a" Jul 06 14:23:15.993042 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.993017 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="ae75ece971c29c3272db012b72745b0b682c3865adecd1fc95d8076db6ec501f" exitCode=0 Jul 06 14:23:15.993162 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:15.993074 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"ae75ece971c29c3272db012b72745b0b682c3865adecd1fc95d8076db6ec501f"} Jul 06 14:23:16.008270 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:16.008234 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:16.008626 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:16.008608 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:17.000266 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.000239 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:23:17.000749 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.000679 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" event={"ID":"23e7f036-b2b0-499a-a66e-fa1c90fc05dc","Type":"ContainerStarted","Data":"e6f8eab37dde100370753229a4f9752e1ec890199388f6c49ded3eebad44fbd6"} Jul 06 14:23:17.031429 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.031359 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" podStartSLOduration=11.143120525 podStartE2EDuration="31.031335725s" totalImagesPullingTime="19.8882152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:17.030528874 +0000 UTC m=+31.801551208" watchObservedRunningTime="2026-07-06 14:23:17.031335725 +0000 UTC m=+31.802358048" Jul 06 14:23:17.494673 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.494642 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q979"] Jul 06 14:23:17.494846 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.494783 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:17.494923 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:17.494871 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:17.498439 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.498416 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvwjb"] Jul 06 14:23:17.498531 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.498506 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:17.498596 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:17.498580 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:17.509014 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.508982 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z8xcc"] Jul 06 14:23:17.509133 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:17.509085 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:17.509245 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:17.509182 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:18.748945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:18.748907 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:18.749305 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:18.749019 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:19.371432 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:19.371391 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:19.371594 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:19.371456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:19.371594 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371536 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:23:19.371726 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371602 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs podName:c1f449df-335a-4ac0-8eeb-6ad99c962ce7 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:51.371585366 +0000 UTC m=+66.142607674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs") pod "network-metrics-daemon-gvwjb" (UID: "c1f449df-335a-4ac0-8eeb-6ad99c962ce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:23:19.371726 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371602 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:23:19.371726 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371626 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:23:19.371726 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371639 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xnf6x for pod openshift-network-diagnostics/network-check-target-z8xcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:23:19.371726 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.371691 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x podName:f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc nodeName:}" failed. No retries permitted until 2026-07-06 14:23:51.37167556 +0000 UTC m=+66.142697873 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xnf6x" (UniqueName: "kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x") pod "network-check-target-z8xcc" (UID: "f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:23:19.471999 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:19.471958 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:19.472165 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.472137 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:23:19.472225 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.472215 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret podName:b1f8d040-e6ef-4596-a93e-b22ff5976566 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:51.472195485 +0000 UTC m=+66.243217806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret") pod "global-pull-secret-syncer-7q979" (UID: "b1f8d040-e6ef-4596-a93e-b22ff5976566") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:23:19.748963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:19.748881 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:19.749406 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.749021 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:19.749406 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:19.749102 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:19.749406 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:19.749237 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:20.748368 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:20.748172 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:20.748610 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:20.748446 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:21.748963 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:21.748926 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:21.749426 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:21.748926 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:21.749426 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:21.749065 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z8xcc" podUID="f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc" Jul 06 14:23:21.749426 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:21.749205 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gvwjb" podUID="c1f449df-335a-4ac0-8eeb-6ad99c962ce7" Jul 06 14:23:22.748178 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:22.748148 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:22.748377 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:22.748246 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q979" podUID="b1f8d040-e6ef-4596-a93e-b22ff5976566" Jul 06 14:23:23.018467 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.018373 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="c75d759676a2d32045d7dc4d4b964e2d8135abd685a2eabe4de35f876e9c8423" exitCode=0 Jul 06 14:23:23.018467 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.018421 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"c75d759676a2d32045d7dc4d4b964e2d8135abd685a2eabe4de35f876e9c8423"} Jul 06 14:23:23.545991 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.545962 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-213.ec2.internal" event="NodeReady" Jul 06 14:23:23.546158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.546076 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 14:23:23.587341 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.587289 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:23:23.602546 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.602482 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9k4wk"] Jul 06 14:23:23.602731 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.602703 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.607209 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.606971 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 14:23:23.607378 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.607192 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 14:23:23.607715 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.607694 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-krbgm\"" Jul 06 14:23:23.607862 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.607837 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 14:23:23.611815 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.611788 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 14:23:23.616775 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.616744 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:23:23.616900 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.616791 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4gspr"] Jul 06 14:23:23.616970 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.616953 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.619183 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.619150 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m58h6\"" Jul 06 14:23:23.619183 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.619150 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 14:23:23.619414 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.619399 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 14:23:23.635166 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.635139 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9k4wk"] Jul 06 14:23:23.635305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.635181 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4gspr"] Jul 06 14:23:23.635305 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.635252 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.637257 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.637237 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 14:23:23.637503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.637486 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 14:23:23.637571 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.637538 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zr548\"" Jul 06 14:23:23.637613 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.637598 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 14:23:23.702117 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702081 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702117 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702115 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lvrz\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702136 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702187 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a623a6-4e37-4653-aace-3b63b801e731-tmp-dir\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702222 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702241 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702284 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfcx\" (UniqueName: \"kubernetes.io/projected/16a623a6-4e37-4653-aace-3b63b801e731-kube-api-access-dbfcx\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.702358 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702347 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16a623a6-4e37-4653-aace-3b63b801e731-config-volume\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.702558 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702374 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702558 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702399 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702558 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702428 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.702558 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.702478 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.748037 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.748002 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:23.748177 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.748016 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:23.750690 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.750666 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:23:23.750848 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.750694 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:23:23.750848 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.750666 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbtwb\"" Jul 06 14:23:23.750848 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.750816 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:23:23.750971 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.750845 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wq2g\"" Jul 06 14:23:23.803462 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803382 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803462 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803418 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lvrz\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803462 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803438 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.803462 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803455 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a623a6-4e37-4653-aace-3b63b801e731-tmp-dir\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803475 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66852\" (UniqueName: \"kubernetes.io/projected/931ac551-f06e-4543-ad74-05080b31bee7-kube-api-access-66852\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803492 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803507 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803532 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfcx\" (UniqueName: \"kubernetes.io/projected/16a623a6-4e37-4653-aace-3b63b801e731-kube-api-access-dbfcx\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803556 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/931ac551-f06e-4543-ad74-05080b31bee7-cert\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:23.803580 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803600 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16a623a6-4e37-4653-aace-3b63b801e731-config-volume\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803635 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:23.803656 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:24.303634467 +0000 UTC m=+39.074656790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:23.803698 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:23.803711 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803719 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.803807 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:23.803778 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:24.30376084 +0000 UTC m=+39.074783155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803867 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16a623a6-4e37-4653-aace-3b63b801e731-tmp-dir\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803884 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.803979 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.804220 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16a623a6-4e37-4653-aace-3b63b801e731-config-volume\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.804219 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.804413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.804259 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.804660 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.804578 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.808155 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.808020 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.808281 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.808073 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.811955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.811930 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfcx\" (UniqueName: \"kubernetes.io/projected/16a623a6-4e37-4653-aace-3b63b801e731-kube-api-access-dbfcx\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:23.812411 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.812379 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.812509 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.812492 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lvrz\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:23.904559 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.904525 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66852\" (UniqueName: \"kubernetes.io/projected/931ac551-f06e-4543-ad74-05080b31bee7-kube-api-access-66852\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.904755 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.904569 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/931ac551-f06e-4543-ad74-05080b31bee7-cert\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.908875 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.908848 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/931ac551-f06e-4543-ad74-05080b31bee7-cert\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.913149 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.913126 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66852\" (UniqueName: \"kubernetes.io/projected/931ac551-f06e-4543-ad74-05080b31bee7-kube-api-access-66852\") pod \"ingress-canary-4gspr\" (UID: \"931ac551-f06e-4543-ad74-05080b31bee7\") " pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:23.944422 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:23.944388 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4gspr" Jul 06 14:23:24.024086 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.024053 2552 generic.go:356] "Generic (PLEG): container finished" podID="d1e77941-bbf8-4e5a-b118-61d68b12bdbb" containerID="6172a7d20da21eb34d8642c4dbd27e193eea46b7cb13f568e4a1a9b712aae277" exitCode=0 Jul 06 14:23:24.024553 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.024122 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerDied","Data":"6172a7d20da21eb34d8642c4dbd27e193eea46b7cb13f568e4a1a9b712aae277"} Jul 06 14:23:24.147896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.147861 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4gspr"] Jul 06 14:23:24.152018 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:24.151994 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931ac551_f06e_4543_ad74_05080b31bee7.slice/crio-5a35b7d84bfdf5c14e9fc52f7569e76230bd460c2dd519b8130d6ce0f97c1da7 WatchSource:0}: Error finding container 5a35b7d84bfdf5c14e9fc52f7569e76230bd460c2dd519b8130d6ce0f97c1da7: Status 404 returned error can't find the container with id 5a35b7d84bfdf5c14e9fc52f7569e76230bd460c2dd519b8130d6ce0f97c1da7 Jul 06 14:23:24.308173 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.308093 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:24.308313 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:24.308248 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:24.308313 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:24.308269 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:24.308437 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:24.308313 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:24.308437 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.308252 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:24.308437 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:24.308336 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:25.308305354 +0000 UTC m=+40.079327662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:24.308437 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:24.308402 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:25.308378289 +0000 UTC m=+40.079400616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:24.748914 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.748834 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:24.751039 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:24.751015 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:23:25.031860 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:25.031765 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" event={"ID":"d1e77941-bbf8-4e5a-b118-61d68b12bdbb","Type":"ContainerStarted","Data":"f737dab7d2e0e2ab8215690aa2081137adde1baf917328125268ff1d58feca59"} Jul 06 14:23:25.033085 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:25.033059 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4gspr" event={"ID":"931ac551-f06e-4543-ad74-05080b31bee7","Type":"ContainerStarted","Data":"5a35b7d84bfdf5c14e9fc52f7569e76230bd460c2dd519b8130d6ce0f97c1da7"} Jul 06 14:23:25.053391 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:25.053339 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8wfl4" podStartSLOduration=10.15845928 podStartE2EDuration="40.053306308s" totalImagesPullingTime="29.894847028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:25.053094245 +0000 UTC m=+39.824116575" watchObservedRunningTime="2026-07-06 14:23:25.053306308 +0000 UTC m=+39.824328637" Jul 06 14:23:25.316336 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:25.316291 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:25.316524 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:25.316374 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:25.316524 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:25.316442 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:25.316524 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:25.316507 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:27.316490348 +0000 UTC m=+42.087512666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:25.316698 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:25.316591 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:25.316698 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:25.316612 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:25.316698 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:25.316688 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:27.316668946 +0000 UTC m=+42.087691255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:27.332023 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:27.331782 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:27.332023 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:27.332006 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:27.332475 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:27.331933 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:27.332475 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:27.332096 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:31.332080714 +0000 UTC m=+46.103103025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:27.332475 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:27.332114 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:27.332475 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:27.332125 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:27.332475 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:27.332167 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:31.332152553 +0000 UTC m=+46.103174862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:28.041473 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:28.041425 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4gspr" event={"ID":"931ac551-f06e-4543-ad74-05080b31bee7","Type":"ContainerStarted","Data":"2f90655fdc130d40f86cfe8113e1133078fca51891ce32f200aa8c7d86802e71"} Jul 06 14:23:28.057303 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:28.057254 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4gspr" podStartSLOduration=1.813315054 podStartE2EDuration="5.057239126s" totalImagesPullingTime="3.243924072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:28.056737802 +0000 UTC m=+42.827760148" watchObservedRunningTime="2026-07-06 14:23:28.057239126 +0000 UTC m=+42.828261455" Jul 06 14:23:31.362463 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:31.362422 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:31.362489 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:31.362583 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:31.362594 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:31.362609 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:31.362660 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:39.362641474 +0000 UTC m=+54.133663782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:31.363131 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:31.362695 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:39.362679561 +0000 UTC m=+54.133701872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:33.872044 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.872011 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm"] Jul 06 14:23:33.895496 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.895469 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-272kw"] Jul 06 14:23:33.895649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.895605 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" Jul 06 14:23:33.897841 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.897818 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5tmn4\"" Jul 06 14:23:33.919435 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.919407 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm"] Jul 06 14:23:33.919435 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.919438 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-272kw"] Jul 06 14:23:33.919582 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.919526 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:33.922498 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.922476 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 14:23:33.922498 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.922487 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 14:23:33.922659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.922491 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 14:23:33.922659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.922616 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-46dnv\"" Jul 06 14:23:33.922989 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.922975 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 14:23:33.929838 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.929819 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 14:23:33.970687 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.970654 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8"] Jul 06 14:23:33.985139 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.985107 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bkj22"] Jul 06 14:23:33.985292 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.985230 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:33.987800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.987774 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 14:23:33.987923 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.987817 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p6twp\"" Jul 06 14:23:33.988743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.988722 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:23:33.988862 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:33.988842 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 14:23:34.007457 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.007435 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8"] Jul 06 14:23:34.007588 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.007469 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bkj22"] Jul 06 14:23:34.007588 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.007550 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.009942 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.009922 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 14:23:34.009942 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.009938 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:23:34.010136 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.009957 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 14:23:34.010136 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.009940 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 14:23:34.010521 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.010479 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jtsdb\"" Jul 06 14:23:34.015703 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.015684 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 14:23:34.084172 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084144 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnt5\" (UniqueName: \"kubernetes.io/projected/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-kube-api-access-lsnt5\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.084347 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084190 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084347 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084217 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bede46ad-fef0-4794-8d5f-ffd886f610bf-serving-cert\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084347 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084269 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.084347 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084314 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-tmp\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084359 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-snapshots\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084381 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084398 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4f49\" (UniqueName: \"kubernetes.io/projected/bede46ad-fef0-4794-8d5f-ffd886f610bf-kube-api-access-b4f49\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.084483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.084469 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8b2h\" (UniqueName: \"kubernetes.io/projected/f7138cf6-c83b-4511-a5e8-d3666d06b7ee-kube-api-access-d8b2h\") pod \"network-check-source-989f8b8dc-4hlhm\" (UID: \"f7138cf6-c83b-4511-a5e8-d3666d06b7ee\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" Jul 06 14:23:34.185059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.184978 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.185059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185016 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-tmp\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185038 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-snapshots\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185057 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185080 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4f49\" (UniqueName: \"kubernetes.io/projected/bede46ad-fef0-4794-8d5f-ffd886f610bf-kube-api-access-b4f49\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185135 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-trusted-ca\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:34.185165 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:34.185257 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls podName:c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:34.685235234 +0000 UTC m=+49.456257563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9hwb8" (UID: "c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9") : secret "samples-operator-tls" not found Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185176 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bede46ad-fef0-4794-8d5f-ffd886f610bf-serving-cert\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185370 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62f58d1e-b920-435c-8d01-177a0e80937f-serving-cert\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185400 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb7m5\" (UniqueName: \"kubernetes.io/projected/62f58d1e-b920-435c-8d01-177a0e80937f-kube-api-access-rb7m5\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.185421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185427 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnt5\" (UniqueName: \"kubernetes.io/projected/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-kube-api-access-lsnt5\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.185874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185476 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8b2h\" (UniqueName: \"kubernetes.io/projected/f7138cf6-c83b-4511-a5e8-d3666d06b7ee-kube-api-access-d8b2h\") pod \"network-check-source-989f8b8dc-4hlhm\" (UID: \"f7138cf6-c83b-4511-a5e8-d3666d06b7ee\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" Jul 06 14:23:34.185874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185480 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-tmp\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185557 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-config\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.185874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185583 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.185874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185686 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bede46ad-fef0-4794-8d5f-ffd886f610bf-snapshots\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.186080 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185935 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.186080 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.185966 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bede46ad-fef0-4794-8d5f-ffd886f610bf-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.189260 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.189236 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bede46ad-fef0-4794-8d5f-ffd886f610bf-serving-cert\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.195590 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.195563 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4f49\" (UniqueName: \"kubernetes.io/projected/bede46ad-fef0-4794-8d5f-ffd886f610bf-kube-api-access-b4f49\") pod \"insights-operator-5c847dcfc4-272kw\" (UID: \"bede46ad-fef0-4794-8d5f-ffd886f610bf\") " pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.195747 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.195726 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnt5\" (UniqueName: \"kubernetes.io/projected/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-kube-api-access-lsnt5\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.196055 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.196038 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8b2h\" (UniqueName: \"kubernetes.io/projected/f7138cf6-c83b-4511-a5e8-d3666d06b7ee-kube-api-access-d8b2h\") pod \"network-check-source-989f8b8dc-4hlhm\" (UID: \"f7138cf6-c83b-4511-a5e8-d3666d06b7ee\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" Jul 06 14:23:34.208439 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.208417 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" Jul 06 14:23:34.228609 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.228591 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" Jul 06 14:23:34.286207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.285946 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-trusted-ca\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.286207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.286026 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62f58d1e-b920-435c-8d01-177a0e80937f-serving-cert\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.286207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.286048 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb7m5\" (UniqueName: \"kubernetes.io/projected/62f58d1e-b920-435c-8d01-177a0e80937f-kube-api-access-rb7m5\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.286207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.286088 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-config\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.286853 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.286815 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-config\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.287547 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.287512 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/62f58d1e-b920-435c-8d01-177a0e80937f-trusted-ca\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.289270 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.289250 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62f58d1e-b920-435c-8d01-177a0e80937f-serving-cert\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.296810 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.296756 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb7m5\" (UniqueName: \"kubernetes.io/projected/62f58d1e-b920-435c-8d01-177a0e80937f-kube-api-access-rb7m5\") pod \"console-operator-68b757fb67-bkj22\" (UID: \"62f58d1e-b920-435c-8d01-177a0e80937f\") " pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.316683 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.316464 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:34.389916 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.389800 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm"] Jul 06 14:23:34.393266 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:34.393225 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7138cf6_c83b_4511_a5e8_d3666d06b7ee.slice/crio-ff756b162da45b0200a551c4a391fd4381c3676dfcc403e2091a12ed0765ed65 WatchSource:0}: Error finding container ff756b162da45b0200a551c4a391fd4381c3676dfcc403e2091a12ed0765ed65: Status 404 returned error can't find the container with id ff756b162da45b0200a551c4a391fd4381c3676dfcc403e2091a12ed0765ed65 Jul 06 14:23:34.415883 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.415853 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-272kw"] Jul 06 14:23:34.418894 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:34.418868 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbede46ad_fef0_4794_8d5f_ffd886f610bf.slice/crio-8e7e16c04fd3e6da4dc82fb0ff259b3e005fc41f0f79bb1cb17fd0ba6f6f7ad9 WatchSource:0}: Error finding container 8e7e16c04fd3e6da4dc82fb0ff259b3e005fc41f0f79bb1cb17fd0ba6f6f7ad9: Status 404 returned error can't find the container with id 8e7e16c04fd3e6da4dc82fb0ff259b3e005fc41f0f79bb1cb17fd0ba6f6f7ad9 Jul 06 14:23:34.479840 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.479728 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bkj22"] Jul 06 14:23:34.484100 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:34.484073 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f58d1e_b920_435c_8d01_177a0e80937f.slice/crio-9774773e0d93331569f699c86b247621b2b8766c73f121af760f66ed196f2fc0 WatchSource:0}: Error finding container 9774773e0d93331569f699c86b247621b2b8766c73f121af760f66ed196f2fc0: Status 404 returned error can't find the container with id 9774773e0d93331569f699c86b247621b2b8766c73f121af760f66ed196f2fc0 Jul 06 14:23:34.689669 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:34.689639 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:34.689817 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:34.689800 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:23:34.689862 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:34.689853 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls podName:c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:35.689839128 +0000 UTC m=+50.460861436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9hwb8" (UID: "c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9") : secret "samples-operator-tls" not found Jul 06 14:23:35.060202 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:35.060159 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" event={"ID":"bede46ad-fef0-4794-8d5f-ffd886f610bf","Type":"ContainerStarted","Data":"8e7e16c04fd3e6da4dc82fb0ff259b3e005fc41f0f79bb1cb17fd0ba6f6f7ad9"} Jul 06 14:23:35.061578 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:35.061543 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" event={"ID":"62f58d1e-b920-435c-8d01-177a0e80937f","Type":"ContainerStarted","Data":"9774773e0d93331569f699c86b247621b2b8766c73f121af760f66ed196f2fc0"} Jul 06 14:23:35.062757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:35.062722 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" event={"ID":"f7138cf6-c83b-4511-a5e8-d3666d06b7ee","Type":"ContainerStarted","Data":"ff756b162da45b0200a551c4a391fd4381c3676dfcc403e2091a12ed0765ed65"} Jul 06 14:23:35.697301 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:35.697271 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:35.697491 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:35.697428 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:23:35.697568 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:35.697495 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls podName:c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:37.697475815 +0000 UTC m=+52.468498140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9hwb8" (UID: "c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9") : secret "samples-operator-tls" not found Jul 06 14:23:37.710141 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:37.710105 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:37.710637 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:37.710281 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:23:37.710637 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:37.710389 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls podName:c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:41.710367322 +0000 UTC m=+56.481389631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9hwb8" (UID: "c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9") : secret "samples-operator-tls" not found Jul 06 14:23:39.077385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.077343 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" event={"ID":"62f58d1e-b920-435c-8d01-177a0e80937f","Type":"ContainerStarted","Data":"5a4ffba53684d17a08d073f417a25aa16ad4a62bc7b4d1ed1dcd06f83ecf6c21"} Jul 06 14:23:39.077827 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.077495 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:39.079216 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.079164 2552 patch_prober.go:28] interesting pod/console-operator-68b757fb67-bkj22 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body= Jul 06 14:23:39.079388 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.079213 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podUID="62f58d1e-b920-435c-8d01-177a0e80937f" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" Jul 06 14:23:39.080083 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.079657 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" event={"ID":"f7138cf6-c83b-4511-a5e8-d3666d06b7ee","Type":"ContainerStarted","Data":"77dc6ece3145b1bf2d79f8617d9478cc8656f89543244631b8737531a4a79ba2"} Jul 06 14:23:39.082840 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.082810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" event={"ID":"bede46ad-fef0-4794-8d5f-ffd886f610bf","Type":"ContainerStarted","Data":"7a0bae04687d337e1e14c3bff7a0448c4051a830ea5b08920aa5baf84d9f60d1"} Jul 06 14:23:39.100945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.100891 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podStartSLOduration=1.655230781 podStartE2EDuration="6.10086965s" totalImagesPullingTime="4.445638869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:39.09258015 +0000 UTC m=+53.863602483" watchObservedRunningTime="2026-07-06 14:23:39.10086965 +0000 UTC m=+53.871891981" Jul 06 14:23:39.114781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.114736 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" podStartSLOduration=1.604643469 podStartE2EDuration="6.114719126s" totalImagesPullingTime="4.510075657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:39.114177467 +0000 UTC m=+53.885199798" watchObservedRunningTime="2026-07-06 14:23:39.114719126 +0000 UTC m=+53.885741455" Jul 06 14:23:39.133184 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.133136 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-4hlhm" podStartSLOduration=1.5869074269999999 podStartE2EDuration="6.133118991s" totalImagesPullingTime="4.546211564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:39.132517092 +0000 UTC m=+53.903539423" watchObservedRunningTime="2026-07-06 14:23:39.133118991 +0000 UTC m=+53.904141364" Jul 06 14:23:39.401174 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.401141 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw"] Jul 06 14:23:39.416068 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.416041 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw"] Jul 06 14:23:39.416211 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.416177 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" Jul 06 14:23:39.419640 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.419616 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-whsk6\"" Jul 06 14:23:39.419640 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.419622 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 14:23:39.419925 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.419807 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 14:23:39.424531 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.424508 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:39.424624 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.424613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tttm\" (UniqueName: \"kubernetes.io/projected/8190d330-b0cf-4771-8894-a35c0e43fbb6-kube-api-access-4tttm\") pod \"migrator-d57d58cc9-2wztw\" (UID: \"8190d330-b0cf-4771-8894-a35c0e43fbb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" Jul 06 14:23:39.424684 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:39.424637 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:23:39.424684 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:39.424653 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7867d68665-khm2n: secret "image-registry-tls" not found Jul 06 14:23:39.424684 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.424666 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:39.424796 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:39.424698 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls podName:59d31435-f584-45d8-80a7-885b0116e21f nodeName:}" failed. No retries permitted until 2026-07-06 14:23:55.424685246 +0000 UTC m=+70.195707554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls") pod "image-registry-7867d68665-khm2n" (UID: "59d31435-f584-45d8-80a7-885b0116e21f") : secret "image-registry-tls" not found Jul 06 14:23:39.424796 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:39.424770 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:23:39.424863 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:39.424818 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls podName:16a623a6-4e37-4653-aace-3b63b801e731 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:55.424803305 +0000 UTC m=+70.195825617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls") pod "dns-default-9k4wk" (UID: "16a623a6-4e37-4653-aace-3b63b801e731") : secret "dns-default-metrics-tls" not found Jul 06 14:23:39.525711 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.525671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tttm\" (UniqueName: \"kubernetes.io/projected/8190d330-b0cf-4771-8894-a35c0e43fbb6-kube-api-access-4tttm\") pod \"migrator-d57d58cc9-2wztw\" (UID: \"8190d330-b0cf-4771-8894-a35c0e43fbb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" Jul 06 14:23:39.534436 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.534408 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tttm\" (UniqueName: \"kubernetes.io/projected/8190d330-b0cf-4771-8894-a35c0e43fbb6-kube-api-access-4tttm\") pod \"migrator-d57d58cc9-2wztw\" (UID: \"8190d330-b0cf-4771-8894-a35c0e43fbb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" Jul 06 14:23:39.726836 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.726762 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" Jul 06 14:23:39.867960 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:39.867931 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw"] Jul 06 14:23:39.871229 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:39.871204 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8190d330_b0cf_4771_8894_a35c0e43fbb6.slice/crio-0fb67efb8b20fe3f0c26935f4cfb03d18f14da3c38367b4a96a18924a7b32734 WatchSource:0}: Error finding container 0fb67efb8b20fe3f0c26935f4cfb03d18f14da3c38367b4a96a18924a7b32734: Status 404 returned error can't find the container with id 0fb67efb8b20fe3f0c26935f4cfb03d18f14da3c38367b4a96a18924a7b32734 Jul 06 14:23:40.087356 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.087298 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" event={"ID":"8190d330-b0cf-4771-8894-a35c0e43fbb6","Type":"ContainerStarted","Data":"0fb67efb8b20fe3f0c26935f4cfb03d18f14da3c38367b4a96a18924a7b32734"} Jul 06 14:23:40.088908 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.088882 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/0.log" Jul 06 14:23:40.089013 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.088927 2552 generic.go:356] "Generic (PLEG): container finished" podID="62f58d1e-b920-435c-8d01-177a0e80937f" containerID="5a4ffba53684d17a08d073f417a25aa16ad4a62bc7b4d1ed1dcd06f83ecf6c21" exitCode=255 Jul 06 14:23:40.089104 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.089017 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" event={"ID":"62f58d1e-b920-435c-8d01-177a0e80937f","Type":"ContainerDied","Data":"5a4ffba53684d17a08d073f417a25aa16ad4a62bc7b4d1ed1dcd06f83ecf6c21"} Jul 06 14:23:40.089316 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.089302 2552 scope.go:117] "RemoveContainer" containerID="5a4ffba53684d17a08d073f417a25aa16ad4a62bc7b4d1ed1dcd06f83ecf6c21" Jul 06 14:23:40.326518 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.326477 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xkc2v"] Jul 06 14:23:40.340102 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.340008 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.342969 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.342941 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r6zhh\"" Jul 06 14:23:40.342969 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.342941 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 14:23:40.343242 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.343224 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 14:23:40.343415 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.343337 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xkc2v"] Jul 06 14:23:40.434637 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.434608 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bb9e974-a960-49aa-afac-23abcf7b6165-crio-socket\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.434637 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.434651 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fb8h\" (UniqueName: \"kubernetes.io/projected/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-api-access-7fb8h\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.434874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.434671 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bb9e974-a960-49aa-afac-23abcf7b6165-data-volume\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.434874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.434737 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.434874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.434835 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536055 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536020 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bb9e974-a960-49aa-afac-23abcf7b6165-crio-socket\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536076 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fb8h\" (UniqueName: \"kubernetes.io/projected/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-api-access-7fb8h\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536100 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bb9e974-a960-49aa-afac-23abcf7b6165-data-volume\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536275 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bb9e974-a960-49aa-afac-23abcf7b6165-crio-socket\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536286 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:40.536388 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:40.536450 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:41.03643131 +0000 UTC m=+55.807453634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : secret "insights-runtime-extractor-tls" not found Jul 06 14:23:40.536503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.536450 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bb9e974-a960-49aa-afac-23abcf7b6165-data-volume\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:40.536722 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:40.536534 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:23:40.536722 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:40.536627 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:41.036607735 +0000 UTC m=+55.807630045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : configmap "kube-rbac-proxy" not found Jul 06 14:23:40.544685 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:40.544663 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fb8h\" (UniqueName: \"kubernetes.io/projected/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-api-access-7fb8h\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:41.041021 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.040974 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:41.041213 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.041120 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:23:41.041213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.041119 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:41.041213 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.041203 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:42.041181536 +0000 UTC m=+56.812203861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : configmap "kube-rbac-proxy" not found Jul 06 14:23:41.041373 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.041298 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:23:41.041425 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.041379 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:42.041361321 +0000 UTC m=+56.812383635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : secret "insights-runtime-extractor-tls" not found Jul 06 14:23:41.094663 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.094636 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:23:41.095145 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.095121 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/0.log" Jul 06 14:23:41.095289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.095165 2552 generic.go:356] "Generic (PLEG): container finished" podID="62f58d1e-b920-435c-8d01-177a0e80937f" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" exitCode=255 Jul 06 14:23:41.095289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.095200 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" event={"ID":"62f58d1e-b920-435c-8d01-177a0e80937f","Type":"ContainerDied","Data":"cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b"} Jul 06 14:23:41.095289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.095283 2552 scope.go:117] "RemoveContainer" containerID="5a4ffba53684d17a08d073f417a25aa16ad4a62bc7b4d1ed1dcd06f83ecf6c21" Jul 06 14:23:41.095567 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.095549 2552 scope.go:117] "RemoveContainer" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" Jul 06 14:23:41.095777 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.095750 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bkj22_openshift-console-operator(62f58d1e-b920-435c-8d01-177a0e80937f)\"" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podUID="62f58d1e-b920-435c-8d01-177a0e80937f" Jul 06 14:23:41.747233 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:41.747207 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:41.747427 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.747405 2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 14:23:41.747536 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:41.747517 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls podName:c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:49.747468111 +0000 UTC m=+64.518490426 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-9hwb8" (UID: "c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9") : secret "samples-operator-tls" not found Jul 06 14:23:42.049002 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.048971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:42.049174 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.049078 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:42.049174 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:42.049104 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:23:42.049174 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:42.049171 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:23:42.049279 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:42.049178 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:44.049160593 +0000 UTC m=+58.820182901 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : configmap "kube-rbac-proxy" not found Jul 06 14:23:42.049279 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:42.049208 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:44.049198024 +0000 UTC m=+58.820220332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : secret "insights-runtime-extractor-tls" not found Jul 06 14:23:42.100114 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.100088 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:23:42.100543 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.100463 2552 scope.go:117] "RemoveContainer" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" Jul 06 14:23:42.100653 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:42.100630 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bkj22_openshift-console-operator(62f58d1e-b920-435c-8d01-177a0e80937f)\"" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podUID="62f58d1e-b920-435c-8d01-177a0e80937f" Jul 06 14:23:42.101905 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.101878 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" event={"ID":"8190d330-b0cf-4771-8894-a35c0e43fbb6","Type":"ContainerStarted","Data":"88680356062d61172ae90bf7ebd361c343a54e1ce9a93cac89f5fc97009f8ba4"} Jul 06 14:23:42.102021 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.101921 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" event={"ID":"8190d330-b0cf-4771-8894-a35c0e43fbb6","Type":"ContainerStarted","Data":"93c73cf131e6d17a34bb2e2f32ba859a00e5c031005b50591261c20d38ff8e2f"} Jul 06 14:23:42.133127 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.133081 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2wztw" podStartSLOduration=1.433950397 podStartE2EDuration="3.133065943s" totalImagesPullingTime="1.699115546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:42.132012608 +0000 UTC m=+56.903034938" watchObservedRunningTime="2026-07-06 14:23:42.133065943 +0000 UTC m=+56.904088375" Jul 06 14:23:42.946569 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:42.946540 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kwqqr_3cb3e28a-fffd-47d3-b379-e3bed6b8b03f/dns-node-resolver/0.log" Jul 06 14:23:43.745260 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:43.745230 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dx8bm_c35f45ce-4d40-4bb9-b18e-e6eb9917a1db/node-ca/0.log" Jul 06 14:23:44.064865 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.064828 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:44.065034 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.064888 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:44.065034 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:44.064977 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:23:44.065034 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:44.065006 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:23:44.065170 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:44.065045 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:48.065028489 +0000 UTC m=+62.836050797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : secret "insights-runtime-extractor-tls" not found Jul 06 14:23:44.065170 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:44.065061 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:48.065053897 +0000 UTC m=+62.836076205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : configmap "kube-rbac-proxy" not found Jul 06 14:23:44.317705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.317624 2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:44.318018 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.318004 2552 scope.go:117] "RemoveContainer" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" Jul 06 14:23:44.318168 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:44.318150 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bkj22_openshift-console-operator(62f58d1e-b920-435c-8d01-177a0e80937f)\"" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podUID="62f58d1e-b920-435c-8d01-177a0e80937f" Jul 06 14:23:44.346639 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.346608 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4gspr_931ac551-f06e-4543-ad74-05080b31bee7/serve-healthcheck-canary/0.log" Jul 06 14:23:44.746478 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.746407 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2wztw_8190d330-b0cf-4771-8894-a35c0e43fbb6/migrator/0.log" Jul 06 14:23:44.948561 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:44.948534 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2wztw_8190d330-b0cf-4771-8894-a35c0e43fbb6/graceful-termination/0.log" Jul 06 14:23:48.013793 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:48.013761 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ftg5" Jul 06 14:23:48.097269 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:48.097230 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:48.097491 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:48.097288 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:48.097491 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:48.097400 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:23:48.097600 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:48.097495 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:56.097473406 +0000 UTC m=+70.868495734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : configmap "kube-rbac-proxy" not found Jul 06 14:23:48.098416 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:48.098396 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:23:48.098529 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:48.098456 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls podName:7bb9e974-a960-49aa-afac-23abcf7b6165 nodeName:}" failed. No retries permitted until 2026-07-06 14:23:56.098440163 +0000 UTC m=+70.869462471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xkc2v" (UID: "7bb9e974-a960-49aa-afac-23abcf7b6165") : secret "insights-runtime-extractor-tls" not found Jul 06 14:23:49.077863 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.077825 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:23:49.078242 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.078191 2552 scope.go:117] "RemoveContainer" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" Jul 06 14:23:49.078387 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:23:49.078359 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bkj22_openshift-console-operator(62f58d1e-b920-435c-8d01-177a0e80937f)\"" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" podUID="62f58d1e-b920-435c-8d01-177a0e80937f" Jul 06 14:23:49.816518 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.816460 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:49.818859 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.818827 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-9hwb8\" (UID: \"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:49.897486 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.897452 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p6twp\"" Jul 06 14:23:49.905842 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:49.905816 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" Jul 06 14:23:50.030582 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:50.030553 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8"] Jul 06 14:23:50.128752 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:50.128720 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" event={"ID":"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9","Type":"ContainerStarted","Data":"ca7cf7530e872388f816c43cdde41bd471e274e1b5f5837481405f91164bf5e5"} Jul 06 14:23:51.430592 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.430543 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:51.430592 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.430599 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:51.432960 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.432933 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:23:51.433491 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.433470 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnf6x\" (UniqueName: \"kubernetes.io/projected/f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc-kube-api-access-xnf6x\") pod \"network-check-target-z8xcc\" (UID: \"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc\") " pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:51.443866 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.443833 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1f449df-335a-4ac0-8eeb-6ad99c962ce7-metrics-certs\") pod \"network-metrics-daemon-gvwjb\" (UID: \"c1f449df-335a-4ac0-8eeb-6ad99c962ce7\") " pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:51.531548 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.531515 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:51.533430 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.533413 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:23:51.544173 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.544140 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b1f8d040-e6ef-4596-a93e-b22ff5976566-original-pull-secret\") pod \"global-pull-secret-syncer-7q979\" (UID: \"b1f8d040-e6ef-4596-a93e-b22ff5976566\") " pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:51.660832 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.660799 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fbtwb\"" Jul 06 14:23:51.663978 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.663961 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wq2g\"" Jul 06 14:23:51.669384 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.669368 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:51.672306 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.672291 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gvwjb" Jul 06 14:23:51.760679 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.760648 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q979" Jul 06 14:23:51.810114 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.810049 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z8xcc"] Jul 06 14:23:51.826818 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.826779 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gvwjb"] Jul 06 14:23:51.924750 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:51.924719 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q979"] Jul 06 14:23:52.252817 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:52.252773 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f449df_335a_4ac0_8eeb_6ad99c962ce7.slice/crio-0936a0fbc0e1ee64d4bab1af091fec419de26d87cb34e3b33f9f1f470715096e WatchSource:0}: Error finding container 0936a0fbc0e1ee64d4bab1af091fec419de26d87cb34e3b33f9f1f470715096e: Status 404 returned error can't find the container with id 0936a0fbc0e1ee64d4bab1af091fec419de26d87cb34e3b33f9f1f470715096e Jul 06 14:23:52.253639 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:52.253522 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5437124_ac3f_4cbb_b864_4b3bb6d2d1bc.slice/crio-d388508bc523d655e8d8a84fa509928138339e16358144b8b6776bf3293781b9 WatchSource:0}: Error finding container d388508bc523d655e8d8a84fa509928138339e16358144b8b6776bf3293781b9: Status 404 returned error can't find the container with id d388508bc523d655e8d8a84fa509928138339e16358144b8b6776bf3293781b9 Jul 06 14:23:52.254166 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:52.254141 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f8d040_e6ef_4596_a93e_b22ff5976566.slice/crio-28684a776f75aa74511db0b18bddb786913181de31a681af9c8a84f2a7258caf WatchSource:0}: Error finding container 28684a776f75aa74511db0b18bddb786913181de31a681af9c8a84f2a7258caf: Status 404 returned error can't find the container with id 28684a776f75aa74511db0b18bddb786913181de31a681af9c8a84f2a7258caf Jul 06 14:23:53.140626 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.140588 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q979" event={"ID":"b1f8d040-e6ef-4596-a93e-b22ff5976566","Type":"ContainerStarted","Data":"28684a776f75aa74511db0b18bddb786913181de31a681af9c8a84f2a7258caf"} Jul 06 14:23:53.141989 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.141956 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvwjb" event={"ID":"c1f449df-335a-4ac0-8eeb-6ad99c962ce7","Type":"ContainerStarted","Data":"0936a0fbc0e1ee64d4bab1af091fec419de26d87cb34e3b33f9f1f470715096e"} Jul 06 14:23:53.144081 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.144053 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" event={"ID":"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9","Type":"ContainerStarted","Data":"1961cd943eaa23914caea957c7427614193718b7d5948802ce457937d8990cea"} Jul 06 14:23:53.144214 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.144105 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" event={"ID":"c2bc8a1b-5bc3-4c50-b68a-42d9a4d3c0e9","Type":"ContainerStarted","Data":"eec32ff534363e07579a2e41955e35ffa3836699105d5285e679d88e6a57251a"} Jul 06 14:23:53.146464 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.146423 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z8xcc" event={"ID":"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc","Type":"ContainerStarted","Data":"dfa51b16f4addbd3d6b73e9e8517772932ab3bb5510d4ab2b482b9371cdcb49f"} Jul 06 14:23:53.146593 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.146472 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z8xcc" event={"ID":"f5437124-ac3f-4cbb-b864-4b3bb6d2d1bc","Type":"ContainerStarted","Data":"d388508bc523d655e8d8a84fa509928138339e16358144b8b6776bf3293781b9"} Jul 06 14:23:53.146593 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.146547 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:23:53.161682 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.161627 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-9hwb8" podStartSLOduration=17.951275308 podStartE2EDuration="20.161604653s" totalImagesPullingTime="2.210329345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:53.160924158 +0000 UTC m=+67.931946515" watchObservedRunningTime="2026-07-06 14:23:53.161604653 +0000 UTC m=+67.932626986" Jul 06 14:23:53.180034 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:53.179969 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z8xcc" podStartSLOduration=68.179949659 podStartE2EDuration="1m8.179949659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:53.178219104 +0000 UTC m=+67.949241458" watchObservedRunningTime="2026-07-06 14:23:53.179949659 +0000 UTC m=+67.950971989" Jul 06 14:23:54.151917 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:54.151873 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvwjb" event={"ID":"c1f449df-335a-4ac0-8eeb-6ad99c962ce7","Type":"ContainerStarted","Data":"9829850d9447564fc49a1fa066abdc7a2c2ef445e2f5798d54c302cb9ce50b50"} Jul 06 14:23:55.157233 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.157181 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gvwjb" event={"ID":"c1f449df-335a-4ac0-8eeb-6ad99c962ce7","Type":"ContainerStarted","Data":"a7fd56cfae834b4367a72c225b8c331ffd650e80b33bf4323a7c586806abd29c"} Jul 06 14:23:55.466011 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.465917 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:55.466011 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.465968 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:55.468523 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.468497 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16a623a6-4e37-4653-aace-3b63b801e731-metrics-tls\") pod \"dns-default-9k4wk\" (UID: \"16a623a6-4e37-4653-aace-3b63b801e731\") " pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:55.468736 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.468718 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"image-registry-7867d68665-khm2n\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:55.719647 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.719565 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-krbgm\"" Jul 06 14:23:55.727133 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.727098 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m58h6\"" Jul 06 14:23:55.727366 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.727345 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:23:55.736397 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:55.736366 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9k4wk" Jul 06 14:23:56.172165 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.172136 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:56.172665 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.172181 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:56.172831 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.172793 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bb9e974-a960-49aa-afac-23abcf7b6165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:56.174999 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.174975 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bb9e974-a960-49aa-afac-23abcf7b6165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xkc2v\" (UID: \"7bb9e974-a960-49aa-afac-23abcf7b6165\") " pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:56.255107 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.255076 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r6zhh\"" Jul 06 14:23:56.263784 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:56.263750 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xkc2v" Jul 06 14:23:57.096660 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.095745 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gvwjb" podStartSLOduration=70.371320304 podStartE2EDuration="1m12.095721974s" totalImagesPullingTime="1.72440167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:55.173069283 +0000 UTC m=+69.944091614" watchObservedRunningTime="2026-07-06 14:23:57.095721974 +0000 UTC m=+71.866744304" Jul 06 14:23:57.096660 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.096152 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xkc2v"] Jul 06 14:23:57.099663 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:57.099629 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb9e974_a960_49aa_afac_23abcf7b6165.slice/crio-3d192499836bd9f8ff30710bd18f2db34c5e3e3cdebc706962bc95f95c6bc9d6 WatchSource:0}: Error finding container 3d192499836bd9f8ff30710bd18f2db34c5e3e3cdebc706962bc95f95c6bc9d6: Status 404 returned error can't find the container with id 3d192499836bd9f8ff30710bd18f2db34c5e3e3cdebc706962bc95f95c6bc9d6 Jul 06 14:23:57.114945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.114916 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9k4wk"] Jul 06 14:23:57.121412 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:57.121378 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a623a6_4e37_4653_aace_3b63b801e731.slice/crio-44e4c3739d02f5910ca79fc2f6ceb379c07aacd2410ab4f8bd9a571928d1c008 WatchSource:0}: Error finding container 44e4c3739d02f5910ca79fc2f6ceb379c07aacd2410ab4f8bd9a571928d1c008: Status 404 returned error can't find the container with id 44e4c3739d02f5910ca79fc2f6ceb379c07aacd2410ab4f8bd9a571928d1c008 Jul 06 14:23:57.131850 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.131814 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:23:57.135690 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:23:57.135655 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d31435_f584_45d8_80a7_885b0116e21f.slice/crio-a31807dabbac81bd5f1d23a2cd3d45aaacfa0682877b144b9a7954e9788b8e10 WatchSource:0}: Error finding container a31807dabbac81bd5f1d23a2cd3d45aaacfa0682877b144b9a7954e9788b8e10: Status 404 returned error can't find the container with id a31807dabbac81bd5f1d23a2cd3d45aaacfa0682877b144b9a7954e9788b8e10 Jul 06 14:23:57.166950 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.166911 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7867d68665-khm2n" event={"ID":"59d31435-f584-45d8-80a7-885b0116e21f","Type":"ContainerStarted","Data":"a31807dabbac81bd5f1d23a2cd3d45aaacfa0682877b144b9a7954e9788b8e10"} Jul 06 14:23:57.168049 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.168020 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xkc2v" event={"ID":"7bb9e974-a960-49aa-afac-23abcf7b6165","Type":"ContainerStarted","Data":"3d192499836bd9f8ff30710bd18f2db34c5e3e3cdebc706962bc95f95c6bc9d6"} Jul 06 14:23:57.169638 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.169609 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q979" event={"ID":"b1f8d040-e6ef-4596-a93e-b22ff5976566","Type":"ContainerStarted","Data":"df03e559e41187654c85883240a3375f84e35a5b10a5ca47a55f52704f2453bd"} Jul 06 14:23:57.170689 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.170667 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9k4wk" event={"ID":"16a623a6-4e37-4653-aace-3b63b801e731","Type":"ContainerStarted","Data":"44e4c3739d02f5910ca79fc2f6ceb379c07aacd2410ab4f8bd9a571928d1c008"} Jul 06 14:23:57.185493 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:57.185442 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7q979" podStartSLOduration=66.462922527 podStartE2EDuration="1m11.185424553s" totalImagesPullingTime="4.722502026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:57.184518771 +0000 UTC m=+71.955541098" watchObservedRunningTime="2026-07-06 14:23:57.185424553 +0000 UTC m=+71.956446885" Jul 06 14:23:58.179350 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:58.179155 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7867d68665-khm2n" event={"ID":"59d31435-f584-45d8-80a7-885b0116e21f","Type":"ContainerStarted","Data":"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f"} Jul 06 14:23:58.201493 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:58.201261 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7867d68665-khm2n" podStartSLOduration=72.201239042 podStartE2EDuration="1m12.201239042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:58.198781239 +0000 UTC m=+72.969803570" watchObservedRunningTime="2026-07-06 14:23:58.201239042 +0000 UTC m=+72.972261375" Jul 06 14:23:59.183017 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:23:59.182980 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:24:01.111338 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.111289 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-srx4c"] Jul 06 14:24:01.114699 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.114679 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.116821 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.116800 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 14:24:01.117247 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.117224 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-946d4\"" Jul 06 14:24:01.117383 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.117253 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 14:24:01.121207 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.121183 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4"] Jul 06 14:24:01.124280 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.124259 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln"] Jul 06 14:24:01.124419 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.124402 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.126551 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.126529 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 14:24:01.126837 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.126816 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 14:24:01.126997 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.126966 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 14:24:01.127098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.127023 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 14:24:01.127710 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.127689 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-srx4c"] Jul 06 14:24:01.127811 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.127774 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.128760 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.128737 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fpcjn\"" Jul 06 14:24:01.130040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.129956 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 14:24:01.137962 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.137930 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln"] Jul 06 14:24:01.146497 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.146433 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4"] Jul 06 14:24:01.216596 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216564 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aaae7e4-46b8-419e-97ea-81a3f6087084-klusterlet-config\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.216596 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216598 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.216833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216621 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98931d88-0dc5-4b00-8a2a-01ec2d019c90-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.216833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216648 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm9s2\" (UniqueName: \"kubernetes.io/projected/7aaae7e4-46b8-419e-97ea-81a3f6087084-kube-api-access-tm9s2\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.216833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216744 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98931d88-0dc5-4b00-8a2a-01ec2d019c90-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.216833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216787 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtpvp\" (UniqueName: \"kubernetes.io/projected/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-kube-api-access-rtpvp\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.217009 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.216858 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aaae7e4-46b8-419e-97ea-81a3f6087084-tmp\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.317659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317622 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aaae7e4-46b8-419e-97ea-81a3f6087084-tmp\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.317849 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317694 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aaae7e4-46b8-419e-97ea-81a3f6087084-klusterlet-config\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.317849 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317718 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.317849 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317749 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98931d88-0dc5-4b00-8a2a-01ec2d019c90-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.318027 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317851 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm9s2\" (UniqueName: \"kubernetes.io/projected/7aaae7e4-46b8-419e-97ea-81a3f6087084-kube-api-access-tm9s2\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.318027 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98931d88-0dc5-4b00-8a2a-01ec2d019c90-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.318027 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.317957 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtpvp\" (UniqueName: \"kubernetes.io/projected/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-kube-api-access-rtpvp\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.318173 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.318093 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aaae7e4-46b8-419e-97ea-81a3f6087084-tmp\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.318564 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.318514 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/98931d88-0dc5-4b00-8a2a-01ec2d019c90-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.320805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.320731 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98931d88-0dc5-4b00-8a2a-01ec2d019c90-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-srx4c\" (UID: \"98931d88-0dc5-4b00-8a2a-01ec2d019c90\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.320805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.320755 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.320805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.320762 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7aaae7e4-46b8-419e-97ea-81a3f6087084-klusterlet-config\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.326067 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.326042 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm9s2\" (UniqueName: \"kubernetes.io/projected/7aaae7e4-46b8-419e-97ea-81a3f6087084-kube-api-access-tm9s2\") pod \"klusterlet-addon-workmgr-684588866-hrvln\" (UID: \"7aaae7e4-46b8-419e-97ea-81a3f6087084\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.326265 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.326245 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtpvp\" (UniqueName: \"kubernetes.io/projected/e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc-kube-api-access-rtpvp\") pod \"managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4\" (UID: \"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.426903 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.426860 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" Jul 06 14:24:01.446922 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.446387 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" Jul 06 14:24:01.455641 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.455483 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:01.645031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.644871 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-srx4c"] Jul 06 14:24:01.648858 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:24:01.648805 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98931d88_0dc5_4b00_8a2a_01ec2d019c90.slice/crio-d947c2bb9eeb73f71051db8ab1c8bf317f984d663964f7515ef1e3d878332819 WatchSource:0}: Error finding container d947c2bb9eeb73f71051db8ab1c8bf317f984d663964f7515ef1e3d878332819: Status 404 returned error can't find the container with id d947c2bb9eeb73f71051db8ab1c8bf317f984d663964f7515ef1e3d878332819 Jul 06 14:24:01.670373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.670341 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4"] Jul 06 14:24:01.672941 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:01.672904 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln"] Jul 06 14:24:01.673874 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:24:01.673846 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode63fa9fd_b94b_4dc2_9dbc_04098f8f0ebc.slice/crio-bf5e0d083488a6e82564c908a0db51e5d345e63d017754cbf28394a4b4cf79f1 WatchSource:0}: Error finding container bf5e0d083488a6e82564c908a0db51e5d345e63d017754cbf28394a4b4cf79f1: Status 404 returned error can't find the container with id bf5e0d083488a6e82564c908a0db51e5d345e63d017754cbf28394a4b4cf79f1 Jul 06 14:24:01.676153 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:24:01.676117 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aaae7e4_46b8_419e_97ea_81a3f6087084.slice/crio-0682384c5845f091f775f946f46e210f61f9ecbea474fe2922f2c7faddbeb35e WatchSource:0}: Error finding container 0682384c5845f091f775f946f46e210f61f9ecbea474fe2922f2c7faddbeb35e: Status 404 returned error can't find the container with id 0682384c5845f091f775f946f46e210f61f9ecbea474fe2922f2c7faddbeb35e Jul 06 14:24:02.195243 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.195210 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" event={"ID":"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc","Type":"ContainerStarted","Data":"bf5e0d083488a6e82564c908a0db51e5d345e63d017754cbf28394a4b4cf79f1"} Jul 06 14:24:02.197012 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.196984 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9k4wk" event={"ID":"16a623a6-4e37-4653-aace-3b63b801e731","Type":"ContainerStarted","Data":"47b817d5ceb564f9115290efffa1b9343d9ec9df6dd644882b40ef593585ac6c"} Jul 06 14:24:02.197131 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.197034 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9k4wk" event={"ID":"16a623a6-4e37-4653-aace-3b63b801e731","Type":"ContainerStarted","Data":"8771937852f07c92ca3b6164fb61e711fccd4aaeb25538d7e5a793fe8ddbf924"} Jul 06 14:24:02.197131 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.197065 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9k4wk" Jul 06 14:24:02.198135 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.198088 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" event={"ID":"7aaae7e4-46b8-419e-97ea-81a3f6087084","Type":"ContainerStarted","Data":"0682384c5845f091f775f946f46e210f61f9ecbea474fe2922f2c7faddbeb35e"} Jul 06 14:24:02.199513 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.199489 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xkc2v" event={"ID":"7bb9e974-a960-49aa-afac-23abcf7b6165","Type":"ContainerStarted","Data":"19e9aec561fd58a62a8cf4a733a7c0e6dae2a129a6e2de8cf0219644a9e0ee1f"} Jul 06 14:24:02.200685 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.200666 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" event={"ID":"98931d88-0dc5-4b00-8a2a-01ec2d019c90","Type":"ContainerStarted","Data":"d947c2bb9eeb73f71051db8ab1c8bf317f984d663964f7515ef1e3d878332819"} Jul 06 14:24:02.214075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.214021 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9k4wk" podStartSLOduration=34.941864759 podStartE2EDuration="39.214002002s" totalImagesPullingTime="4.272137243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:02.213141383 +0000 UTC m=+76.984163758" watchObservedRunningTime="2026-07-06 14:24:02.214002002 +0000 UTC m=+76.985024331" Jul 06 14:24:02.748893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:02.748498 2552 scope.go:117] "RemoveContainer" containerID="cd1a9db54dc27a0c3fa856218ab6f06717bcdab1628b6b93d723eac1243b2e1b" Jul 06 14:24:03.209743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:03.209458 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:24:03.209743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:03.209598 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" event={"ID":"62f58d1e-b920-435c-8d01-177a0e80937f","Type":"ContainerStarted","Data":"448341ba5637d5592b5c82553de0777dfab7410113aa001855eb6da4c8a2bbe4"} Jul 06 14:24:03.210566 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:03.210416 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:24:03.214522 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:03.214494 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xkc2v" event={"ID":"7bb9e974-a960-49aa-afac-23abcf7b6165","Type":"ContainerStarted","Data":"c64f287d90a10e848f5cd6c9b79afc2a1f1e1368d4831c9ab9fa976e77236090"} Jul 06 14:24:03.849971 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:03.849901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-bkj22" Jul 06 14:24:04.042992 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.042951 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-ldhwf"] Jul 06 14:24:04.056858 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.056162 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ldhwf"] Jul 06 14:24:04.057059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.057044 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:04.061237 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.060501 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 14:24:04.061237 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.060864 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lffcn\"" Jul 06 14:24:04.061237 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.061093 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 14:24:04.150888 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.150677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jkjn\" (UniqueName: \"kubernetes.io/projected/8298dda6-ef7e-4962-972f-64264e2a3aff-kube-api-access-8jkjn\") pod \"downloads-685bfb5f64-ldhwf\" (UID: \"8298dda6-ef7e-4962-972f-64264e2a3aff\") " pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:04.222609 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.222543 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" event={"ID":"98931d88-0dc5-4b00-8a2a-01ec2d019c90","Type":"ContainerStarted","Data":"51b6f994ffb7f46237862ffb50cc90cc26d1402b0f6df181c4ad1f732172499b"} Jul 06 14:24:04.243067 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.243003 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-srx4c" podStartSLOduration=1.439819494 podStartE2EDuration="3.242978803s" totalImagesPullingTime="1.803159309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:04.239396373 +0000 UTC m=+79.010418705" watchObservedRunningTime="2026-07-06 14:24:04.242978803 +0000 UTC m=+79.014001134" Jul 06 14:24:04.252893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.252184 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jkjn\" (UniqueName: \"kubernetes.io/projected/8298dda6-ef7e-4962-972f-64264e2a3aff-kube-api-access-8jkjn\") pod \"downloads-685bfb5f64-ldhwf\" (UID: \"8298dda6-ef7e-4962-972f-64264e2a3aff\") " pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:04.266501 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.266437 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jkjn\" (UniqueName: \"kubernetes.io/projected/8298dda6-ef7e-4962-972f-64264e2a3aff-kube-api-access-8jkjn\") pod \"downloads-685bfb5f64-ldhwf\" (UID: \"8298dda6-ef7e-4962-972f-64264e2a3aff\") " pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:04.371703 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:04.371670 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:07.766079 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:07.766045 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ldhwf"] Jul 06 14:24:07.770968 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:24:07.770923 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8298dda6_ef7e_4962_972f_64264e2a3aff.slice/crio-2b67a22af958fd508edb9f4f27e620d403e9ecc7abc6ab7295c83656016c2ef2 WatchSource:0}: Error finding container 2b67a22af958fd508edb9f4f27e620d403e9ecc7abc6ab7295c83656016c2ef2: Status 404 returned error can't find the container with id 2b67a22af958fd508edb9f4f27e620d403e9ecc7abc6ab7295c83656016c2ef2 Jul 06 14:24:08.240208 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.240082 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" event={"ID":"e63fa9fd-b94b-4dc2-9dbc-04098f8f0ebc","Type":"ContainerStarted","Data":"c32e560a0bf38a639f6b672e9f53e01464540fdcf42b2ccebfc096b8fa566ce2"} Jul 06 14:24:08.241360 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.241311 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ldhwf" event={"ID":"8298dda6-ef7e-4962-972f-64264e2a3aff","Type":"ContainerStarted","Data":"2b67a22af958fd508edb9f4f27e620d403e9ecc7abc6ab7295c83656016c2ef2"} Jul 06 14:24:08.242708 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.242680 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" event={"ID":"7aaae7e4-46b8-419e-97ea-81a3f6087084","Type":"ContainerStarted","Data":"6d5b05fb7d4af0eb7bdaf91f763707b57ac6fc2494a1d7e906bf3713a0ea0060"} Jul 06 14:24:08.242825 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.242806 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:08.244668 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.244650 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" Jul 06 14:24:08.244989 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.244967 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xkc2v" event={"ID":"7bb9e974-a960-49aa-afac-23abcf7b6165","Type":"ContainerStarted","Data":"2fabfed98fd82acb8aa6dc32422c13b1e9874923af68346d7bf5fee52b25dfbe"} Jul 06 14:24:08.255044 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.254993 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-569dcdc6c4-nnwr4" podStartSLOduration=1.314509942 podStartE2EDuration="7.254979224s" totalImagesPullingTime="5.940469282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:08.254599834 +0000 UTC m=+83.025622166" watchObservedRunningTime="2026-07-06 14:24:08.254979224 +0000 UTC m=+83.026001553" Jul 06 14:24:08.270285 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.270225 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-684588866-hrvln" podStartSLOduration=1.310571136 podStartE2EDuration="7.270207028s" totalImagesPullingTime="5.959635892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:08.268618616 +0000 UTC m=+83.039640948" watchObservedRunningTime="2026-07-06 14:24:08.270207028 +0000 UTC m=+83.041229358" Jul 06 14:24:08.285122 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:08.285062 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xkc2v" podStartSLOduration=17.95186103 podStartE2EDuration="28.285043919s" totalImagesPullingTime="10.333182889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:23:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:08.283893023 +0000 UTC m=+83.054915355" watchObservedRunningTime="2026-07-06 14:24:08.285043919 +0000 UTC m=+83.056066249" Jul 06 14:24:12.217054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:12.217027 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9k4wk" Jul 06 14:24:15.732656 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:15.732617 2552 patch_prober.go:28] interesting pod/image-registry-7867d68665-khm2n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 14:24:15.733229 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:15.732688 2552 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7867d68665-khm2n" podUID="59d31435-f584-45d8-80a7-885b0116e21f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:24:20.191085 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:20.191058 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:24:21.860665 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.860628 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f8fdr"] Jul 06 14:24:21.863762 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.863739 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.865853 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.865833 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 14:24:21.866499 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.866472 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 14:24:21.866499 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.866492 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 14:24:21.866667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.866627 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 14:24:21.867248 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.867230 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k77mc\"" Jul 06 14:24:21.867383 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.867298 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 14:24:21.867531 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.867513 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 14:24:21.898671 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898637 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.898671 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898673 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.898906 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-root\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.898906 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898847 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-textfile\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.898906 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898887 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-sys\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.899038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898919 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkfk5\" (UniqueName: \"kubernetes.io/projected/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-kube-api-access-qkfk5\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.899038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898954 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-metrics-client-ca\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.899038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.898986 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-wtmp\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.899038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.899015 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999538 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999493 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999715 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999550 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999715 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999608 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-root\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999715 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999635 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-textfile\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999715 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:24:21.999660 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 14:24:21.999715 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999709 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-sys\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999711 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-root\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999666 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-sys\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:24:21.999744 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls podName:b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe nodeName:}" failed. No retries permitted until 2026-07-06 14:24:22.499724112 +0000 UTC m=+97.270746437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls") pod "node-exporter-f8fdr" (UID: "b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe") : secret "node-exporter-tls" not found Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999799 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkfk5\" (UniqueName: \"kubernetes.io/projected/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-kube-api-access-qkfk5\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999842 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-metrics-client-ca\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999871 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-wtmp\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:21.999987 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:21.999905 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.000373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.000011 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-textfile\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.000373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.000080 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-wtmp\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.000559 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.000532 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-metrics-client-ca\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.000628 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.000566 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.002681 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.002659 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.010371 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.010345 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkfk5\" (UniqueName: \"kubernetes.io/projected/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-kube-api-access-qkfk5\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.503343 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.503283 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.506065 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.506028 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe-node-exporter-tls\") pod \"node-exporter-f8fdr\" (UID: \"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe\") " pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:22.775369 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:22.775258 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f8fdr" Jul 06 14:24:24.012635 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:24.012600 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:24:24.165200 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:24.165162 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z8xcc" Jul 06 14:24:28.301823 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:24:28.301794 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fba76a_fb9b_4a4c_a2f5_74e55c217dbe.slice/crio-2d463de915e867adf111b394d1cd7cf40cc26e8ed6308ada259e014f595bda31 WatchSource:0}: Error finding container 2d463de915e867adf111b394d1cd7cf40cc26e8ed6308ada259e014f595bda31: Status 404 returned error can't find the container with id 2d463de915e867adf111b394d1cd7cf40cc26e8ed6308ada259e014f595bda31 Jul 06 14:24:28.324962 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:28.324927 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8fdr" event={"ID":"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe","Type":"ContainerStarted","Data":"2d463de915e867adf111b394d1cd7cf40cc26e8ed6308ada259e014f595bda31"} Jul 06 14:24:29.331966 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:29.331724 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ldhwf" event={"ID":"8298dda6-ef7e-4962-972f-64264e2a3aff","Type":"ContainerStarted","Data":"2f7f8f33335aafb0b79383cfc54c97043afaab3a0b8bf86b88ad5ba1c51dbf13"} Jul 06 14:24:29.332540 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:29.332467 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:29.334757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:29.334235 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8fdr" event={"ID":"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe","Type":"ContainerStarted","Data":"b3e5d5a668366001f92c5a7c253dadcd310e6af5ac270de5bceb37bcb6bbb4e6"} Jul 06 14:24:29.342220 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:29.342195 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-ldhwf" Jul 06 14:24:29.351652 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:29.351595 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-ldhwf" podStartSLOduration=4.493344464 podStartE2EDuration="25.35157559s" totalImagesPullingTime="20.858231126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:29.351458276 +0000 UTC m=+104.122480607" watchObservedRunningTime="2026-07-06 14:24:29.35157559 +0000 UTC m=+104.122597922" Jul 06 14:24:30.342051 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:30.342010 2552 generic.go:356] "Generic (PLEG): container finished" podID="b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe" containerID="b3e5d5a668366001f92c5a7c253dadcd310e6af5ac270de5bceb37bcb6bbb4e6" exitCode=0 Jul 06 14:24:30.342548 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:30.342111 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8fdr" event={"ID":"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe","Type":"ContainerDied","Data":"b3e5d5a668366001f92c5a7c253dadcd310e6af5ac270de5bceb37bcb6bbb4e6"} Jul 06 14:24:31.349599 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:31.349555 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8fdr" event={"ID":"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe","Type":"ContainerStarted","Data":"d6bbd5bdbb210aafe6961ecfecfa13727038104f383514de032976221e6e10c0"} Jul 06 14:24:31.350067 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:31.349647 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f8fdr" event={"ID":"b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe","Type":"ContainerStarted","Data":"baae538780d16672dc31d685705d7e3cd81ece98882f634d1a7ae874f6a44232"} Jul 06 14:24:49.036145 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.036103 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7867d68665-khm2n" podUID="59d31435-f584-45d8-80a7-885b0116e21f" containerName="registry" containerID="cri-o://17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f" gracePeriod=30 Jul 06 14:24:49.311054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.310988 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:24:49.332896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.332831 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f8fdr" podStartSLOduration=27.354831244 podStartE2EDuration="28.332813439s" totalImagesPullingTime="895.922249ms" totalInitContainerRuntime="82.059946ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:31.366523448 +0000 UTC m=+106.137545806" watchObservedRunningTime="2026-07-06 14:24:49.332813439 +0000 UTC m=+124.103835789" Jul 06 14:24:49.416827 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.416789 2552 generic.go:356] "Generic (PLEG): container finished" podID="59d31435-f584-45d8-80a7-885b0116e21f" containerID="17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f" exitCode=0 Jul 06 14:24:49.416827 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.416832 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7867d68665-khm2n" Jul 06 14:24:49.417040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.416837 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7867d68665-khm2n" event={"ID":"59d31435-f584-45d8-80a7-885b0116e21f","Type":"ContainerDied","Data":"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f"} Jul 06 14:24:49.417040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.416885 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7867d68665-khm2n" event={"ID":"59d31435-f584-45d8-80a7-885b0116e21f","Type":"ContainerDied","Data":"a31807dabbac81bd5f1d23a2cd3d45aaacfa0682877b144b9a7954e9788b8e10"} Jul 06 14:24:49.417040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.416908 2552 scope.go:117] "RemoveContainer" containerID="17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f" Jul 06 14:24:49.424646 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.424630 2552 scope.go:117] "RemoveContainer" containerID="17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f" Jul 06 14:24:49.424914 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:24:49.424895 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f\": container with ID starting with 17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f not found: ID does not exist" containerID="17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f" Jul 06 14:24:49.424988 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.424923 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f"} err="failed to get container status \"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f\": rpc error: code = NotFound desc = could not find container \"17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f\": container with ID starting with 17f06f969f799e09dad367f9dee83223ffb4a27f758f58ee72328b6be4cdc36f not found: ID does not exist" Jul 06 14:24:49.431170 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431154 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431228 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431191 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431263 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431249 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431302 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431283 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431394 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431374 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431449 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431409 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lvrz\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431503 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431468 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431563 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431511 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") pod \"59d31435-f584-45d8-80a7-885b0116e21f\" (UID: \"59d31435-f584-45d8-80a7-885b0116e21f\") " Jul 06 14:24:49.431825 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.431772 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:24:49.432163 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.432137 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:24:49.433865 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.433822 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:24:49.433960 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.433908 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:24:49.433960 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.433945 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:49.434070 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.434029 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz" (OuterVolumeSpecName: "kube-api-access-4lvrz") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "kube-api-access-4lvrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:49.434070 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.434054 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:49.440982 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.440948 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "59d31435-f584-45d8-80a7-885b0116e21f" (UID: "59d31435-f584-45d8-80a7-885b0116e21f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:24:49.533050 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533013 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-image-registry-private-configuration\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533050 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533043 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59d31435-f584-45d8-80a7-885b0116e21f-installation-pull-secrets\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533050 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533052 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59d31435-f584-45d8-80a7-885b0116e21f-ca-trust-extracted\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533050 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533062 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-bound-sa-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533308 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533071 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-trusted-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533308 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533080 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lvrz\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-kube-api-access-4lvrz\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533308 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533089 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59d31435-f584-45d8-80a7-885b0116e21f-registry-certificates\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.533308 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.533096 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59d31435-f584-45d8-80a7-885b0116e21f-registry-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:24:49.737643 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.737613 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:24:49.741905 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.741880 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7867d68665-khm2n"] Jul 06 14:24:49.752745 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:24:49.752719 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d31435-f584-45d8-80a7-885b0116e21f" path="/var/lib/kubelet/pods/59d31435-f584-45d8-80a7-885b0116e21f/volumes" Jul 06 14:25:10.494756 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:25:10.494722 2552 generic.go:356] "Generic (PLEG): container finished" podID="bede46ad-fef0-4794-8d5f-ffd886f610bf" containerID="7a0bae04687d337e1e14c3bff7a0448c4051a830ea5b08920aa5baf84d9f60d1" exitCode=0 Jul 06 14:25:10.495159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:25:10.494800 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" event={"ID":"bede46ad-fef0-4794-8d5f-ffd886f610bf","Type":"ContainerDied","Data":"7a0bae04687d337e1e14c3bff7a0448c4051a830ea5b08920aa5baf84d9f60d1"} Jul 06 14:25:10.495159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:25:10.495139 2552 scope.go:117] "RemoveContainer" containerID="7a0bae04687d337e1e14c3bff7a0448c4051a830ea5b08920aa5baf84d9f60d1" Jul 06 14:25:11.500267 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:25:11.500236 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-272kw" event={"ID":"bede46ad-fef0-4794-8d5f-ffd886f610bf","Type":"ContainerStarted","Data":"8843f59f1fec092bdfb2aa64a661cafdc75e6152b73077ffaac5b7ac088c1e18"} Jul 06 14:27:45.621962 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:45.621926 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:27:45.623702 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:45.623681 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:27:45.626335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:45.626306 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:27:45.628664 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:45.628644 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:27:45.632913 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:45.632882 2552 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 14:27:46.244649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.244616 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v"] Jul 06 14:27:46.245700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.244871 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d31435-f584-45d8-80a7-885b0116e21f" containerName="registry" Jul 06 14:27:46.245700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.244881 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d31435-f584-45d8-80a7-885b0116e21f" containerName="registry" Jul 06 14:27:46.245700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.244934 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d31435-f584-45d8-80a7-885b0116e21f" containerName="registry" Jul 06 14:27:46.246586 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.246570 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.248611 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.248586 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:27:46.248740 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.248628 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:27:46.249138 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.249119 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8xn2\"" Jul 06 14:27:46.256273 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.256252 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v"] Jul 06 14:27:46.305707 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.305673 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.305872 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.305715 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw59q\" (UniqueName: \"kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.305872 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.305757 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.407043 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.407003 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.407164 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.407059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.407164 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.407087 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw59q\" (UniqueName: \"kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.407443 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.407426 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.407491 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.407459 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.414639 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.414611 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw59q\" (UniqueName: \"kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.555990 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.555960 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:27:46.695026 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:46.694995 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v"] Jul 06 14:27:46.698492 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:27:46.698464 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb2662e_2487_4f0d_bf39_6999821cf12f.slice/crio-21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715 WatchSource:0}: Error finding container 21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715: Status 404 returned error can't find the container with id 21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715 Jul 06 14:27:47.058906 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:47.058876 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" event={"ID":"efb2662e-2487-4f0d-bf39-6999821cf12f","Type":"ContainerStarted","Data":"21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715"} Jul 06 14:27:55.090867 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:55.090830 2552 generic.go:356] "Generic (PLEG): container finished" podID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerID="c3c18354574051ba78d597ed659b1a53670a5c01c1031975abe16a621738d3e5" exitCode=0 Jul 06 14:27:55.091275 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:55.090893 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" event={"ID":"efb2662e-2487-4f0d-bf39-6999821cf12f","Type":"ContainerDied","Data":"c3c18354574051ba78d597ed659b1a53670a5c01c1031975abe16a621738d3e5"} Jul 06 14:27:55.091873 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:55.091854 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:27:58.104109 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:58.104073 2552 generic.go:356] "Generic (PLEG): container finished" podID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerID="9d0989f0bdf170d26dfe6e8015f2ccfdaa469b9155a294d234e658c3635f4564" exitCode=0 Jul 06 14:27:58.104544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:27:58.104164 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" event={"ID":"efb2662e-2487-4f0d-bf39-6999821cf12f","Type":"ContainerDied","Data":"9d0989f0bdf170d26dfe6e8015f2ccfdaa469b9155a294d234e658c3635f4564"} Jul 06 14:28:08.146058 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:08.146020 2552 generic.go:356] "Generic (PLEG): container finished" podID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerID="c09cff55230f6843ce7f4a35614be7997a32255ba6041b8420b4adf146ba6849" exitCode=0 Jul 06 14:28:08.146477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:08.146104 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" event={"ID":"efb2662e-2487-4f0d-bf39-6999821cf12f","Type":"ContainerDied","Data":"c09cff55230f6843ce7f4a35614be7997a32255ba6041b8420b4adf146ba6849"} Jul 06 14:28:09.293240 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.293213 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:28:09.392230 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.392195 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle\") pod \"efb2662e-2487-4f0d-bf39-6999821cf12f\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " Jul 06 14:28:09.392230 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.392235 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw59q\" (UniqueName: \"kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q\") pod \"efb2662e-2487-4f0d-bf39-6999821cf12f\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " Jul 06 14:28:09.392448 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.392259 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util\") pod \"efb2662e-2487-4f0d-bf39-6999821cf12f\" (UID: \"efb2662e-2487-4f0d-bf39-6999821cf12f\") " Jul 06 14:28:09.392874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.392837 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle" (OuterVolumeSpecName: "bundle") pod "efb2662e-2487-4f0d-bf39-6999821cf12f" (UID: "efb2662e-2487-4f0d-bf39-6999821cf12f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:09.394447 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.394412 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q" (OuterVolumeSpecName: "kube-api-access-mw59q") pod "efb2662e-2487-4f0d-bf39-6999821cf12f" (UID: "efb2662e-2487-4f0d-bf39-6999821cf12f"). InnerVolumeSpecName "kube-api-access-mw59q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:28:09.397213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.397188 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util" (OuterVolumeSpecName: "util") pod "efb2662e-2487-4f0d-bf39-6999821cf12f" (UID: "efb2662e-2487-4f0d-bf39-6999821cf12f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:09.493234 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.493141 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw59q\" (UniqueName: \"kubernetes.io/projected/efb2662e-2487-4f0d-bf39-6999821cf12f-kube-api-access-mw59q\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:09.493234 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.493172 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:09.493234 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:09.493183 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb2662e-2487-4f0d-bf39-6999821cf12f-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:10.154886 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:10.154854 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" event={"ID":"efb2662e-2487-4f0d-bf39-6999821cf12f","Type":"ContainerDied","Data":"21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715"} Jul 06 14:28:10.154886 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:10.154881 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxl7v" Jul 06 14:28:10.155086 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:10.154899 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21853a880cc21ab205309ae58f06042578168efa4ceebbb502bbe753a2dd5715" Jul 06 14:28:14.426197 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.426161 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns"] Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.426948 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="util" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.426977 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="util" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.427004 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="extract" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.427014 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="extract" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.427025 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="pull" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.427035 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="pull" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.427177 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb2662e-2487-4f0d-bf39-6999821cf12f" containerName="extract" Jul 06 14:28:14.431126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.430100 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.432838 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.432813 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:28:14.432979 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.432911 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lfbwj\"" Jul 06 14:28:14.433039 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.432990 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 14:28:14.443044 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.443014 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns"] Jul 06 14:28:14.532362 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.532311 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nskjg\" (UniqueName: \"kubernetes.io/projected/1c177ae9-33ac-4baa-8944-a49283958994-kube-api-access-nskjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.532362 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.532376 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c177ae9-33ac-4baa-8944-a49283958994-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.633063 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.633013 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nskjg\" (UniqueName: \"kubernetes.io/projected/1c177ae9-33ac-4baa-8944-a49283958994-kube-api-access-nskjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.633245 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.633083 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c177ae9-33ac-4baa-8944-a49283958994-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.633454 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.633433 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c177ae9-33ac-4baa-8944-a49283958994-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.641685 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.641656 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nskjg\" (UniqueName: \"kubernetes.io/projected/1c177ae9-33ac-4baa-8944-a49283958994-kube-api-access-nskjg\") pod \"cert-manager-operator-controller-manager-55c444bf86-4gdns\" (UID: \"1c177ae9-33ac-4baa-8944-a49283958994\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.739251 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.739141 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" Jul 06 14:28:14.879005 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:14.878969 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns"] Jul 06 14:28:14.883685 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:14.883652 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c177ae9_33ac_4baa_8944_a49283958994.slice/crio-a332ce59bc502824b34007196bf7431083a9e3aa5d7a974bd053954567782abd WatchSource:0}: Error finding container a332ce59bc502824b34007196bf7431083a9e3aa5d7a974bd053954567782abd: Status 404 returned error can't find the container with id a332ce59bc502824b34007196bf7431083a9e3aa5d7a974bd053954567782abd Jul 06 14:28:15.173008 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:15.172969 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" event={"ID":"1c177ae9-33ac-4baa-8944-a49283958994","Type":"ContainerStarted","Data":"a332ce59bc502824b34007196bf7431083a9e3aa5d7a974bd053954567782abd"} Jul 06 14:28:17.182054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:17.182017 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" event={"ID":"1c177ae9-33ac-4baa-8944-a49283958994","Type":"ContainerStarted","Data":"06bee1a705080139ced9e959caccbb8ddae325e3f2caced189df12329ffd2264"} Jul 06 14:28:17.213537 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:17.213467 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4gdns" podStartSLOduration=1.70776312 podStartE2EDuration="3.213443111s" totalImagesPullingTime="1.505679991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:17.211423072 +0000 UTC m=+331.982445403" watchObservedRunningTime="2026-07-06 14:28:17.213443111 +0000 UTC m=+331.984465443" Jul 06 14:28:18.646893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.646857 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9"] Jul 06 14:28:18.650269 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.650252 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.654243 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.654220 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8xn2\"" Jul 06 14:28:18.654386 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.654371 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:28:18.654638 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.654623 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:28:18.664512 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.664484 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9"] Jul 06 14:28:18.766651 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.766613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzkn6\" (UniqueName: \"kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.766837 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.766658 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.766837 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.766743 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.868101 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.868046 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzkn6\" (UniqueName: \"kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.868101 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.868107 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.868455 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.868139 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.868584 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.868566 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.868626 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.868566 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.879638 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.879606 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzkn6\" (UniqueName: \"kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:18.959649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:18.959551 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:19.094523 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:19.094489 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9"] Jul 06 14:28:19.098004 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:19.097975 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca05806_e559_459b_9e68_5f1ab17ce821.slice/crio-237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c WatchSource:0}: Error finding container 237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c: Status 404 returned error can't find the container with id 237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c Jul 06 14:28:19.191519 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:19.191473 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerStarted","Data":"3f4b0bbd99ea306f4f077db712b97cd22521e39eec96f15f1e41ac360e72afe9"} Jul 06 14:28:19.191692 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:19.191540 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerStarted","Data":"237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c"} Jul 06 14:28:20.197171 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.197072 2552 generic.go:356] "Generic (PLEG): container finished" podID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerID="3f4b0bbd99ea306f4f077db712b97cd22521e39eec96f15f1e41ac360e72afe9" exitCode=0 Jul 06 14:28:20.197569 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.197166 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerDied","Data":"3f4b0bbd99ea306f4f077db712b97cd22521e39eec96f15f1e41ac360e72afe9"} Jul 06 14:28:20.993151 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.993116 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz"] Jul 06 14:28:20.996446 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.996418 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:20.998543 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.998515 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8phqm\"" Jul 06 14:28:20.998543 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.998515 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 14:28:20.998972 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:20.998909 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 14:28:21.008529 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.008501 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz"] Jul 06 14:28:21.087106 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.087069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.087300 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.087153 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znq8z\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-kube-api-access-znq8z\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.188519 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.188480 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.188721 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.188567 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znq8z\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-kube-api-access-znq8z\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.199972 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.199943 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.200425 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.200156 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znq8z\" (UniqueName: \"kubernetes.io/projected/33cc2962-3136-4d42-94ae-5c6fbc636582-kube-api-access-znq8z\") pod \"cert-manager-cainjector-559cdd8c87-zz4bz\" (UID: \"33cc2962-3136-4d42-94ae-5c6fbc636582\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:21.306656 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:21.306621 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" Jul 06 14:28:22.140099 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:22.140065 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz"] Jul 06 14:28:22.143718 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:22.143691 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33cc2962_3136_4d42_94ae_5c6fbc636582.slice/crio-0cfefc57e324710d28bbc5a3edc893839779d51478943ae5216b2f5d5830d55c WatchSource:0}: Error finding container 0cfefc57e324710d28bbc5a3edc893839779d51478943ae5216b2f5d5830d55c: Status 404 returned error can't find the container with id 0cfefc57e324710d28bbc5a3edc893839779d51478943ae5216b2f5d5830d55c Jul 06 14:28:22.206297 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:22.206259 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" event={"ID":"33cc2962-3136-4d42-94ae-5c6fbc636582","Type":"ContainerStarted","Data":"0cfefc57e324710d28bbc5a3edc893839779d51478943ae5216b2f5d5830d55c"} Jul 06 14:28:22.208123 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:22.208097 2552 generic.go:356] "Generic (PLEG): container finished" podID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerID="c017312434d29f50e31c792d5d375e4ee0c9db340835c43d281bb78218a3e9e0" exitCode=0 Jul 06 14:28:22.208240 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:22.208138 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerDied","Data":"c017312434d29f50e31c792d5d375e4ee0c9db340835c43d281bb78218a3e9e0"} Jul 06 14:28:23.215524 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:23.215488 2552 generic.go:356] "Generic (PLEG): container finished" podID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerID="5cfd3be586bae793d3f88c019debd75d54f79faf0e452a7f82812afbca0c77a0" exitCode=0 Jul 06 14:28:23.215940 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:23.215558 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerDied","Data":"5cfd3be586bae793d3f88c019debd75d54f79faf0e452a7f82812afbca0c77a0"} Jul 06 14:28:24.678776 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.678748 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:24.817936 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.817901 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzkn6\" (UniqueName: \"kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6\") pod \"4ca05806-e559-459b-9e68-5f1ab17ce821\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " Jul 06 14:28:24.817936 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.817940 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle\") pod \"4ca05806-e559-459b-9e68-5f1ab17ce821\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " Jul 06 14:28:24.818181 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.817972 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util\") pod \"4ca05806-e559-459b-9e68-5f1ab17ce821\" (UID: \"4ca05806-e559-459b-9e68-5f1ab17ce821\") " Jul 06 14:28:24.818362 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.818332 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle" (OuterVolumeSpecName: "bundle") pod "4ca05806-e559-459b-9e68-5f1ab17ce821" (UID: "4ca05806-e559-459b-9e68-5f1ab17ce821"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:24.820078 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.820034 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6" (OuterVolumeSpecName: "kube-api-access-gzkn6") pod "4ca05806-e559-459b-9e68-5f1ab17ce821" (UID: "4ca05806-e559-459b-9e68-5f1ab17ce821"). InnerVolumeSpecName "kube-api-access-gzkn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:28:24.822028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.822002 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util" (OuterVolumeSpecName: "util") pod "4ca05806-e559-459b-9e68-5f1ab17ce821" (UID: "4ca05806-e559-459b-9e68-5f1ab17ce821"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:24.919343 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.919223 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzkn6\" (UniqueName: \"kubernetes.io/projected/4ca05806-e559-459b-9e68-5f1ab17ce821-kube-api-access-gzkn6\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:24.919343 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.919258 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:24.919343 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:24.919269 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ca05806-e559-459b-9e68-5f1ab17ce821-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:25.225661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:25.225554 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" event={"ID":"33cc2962-3136-4d42-94ae-5c6fbc636582","Type":"ContainerStarted","Data":"a6a3466b8c63c8f61c13fe4c423ba71a40392cee284051db55daf1c644ce4371"} Jul 06 14:28:25.227718 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:25.227695 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" Jul 06 14:28:25.227852 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:25.227688 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmz7w9" event={"ID":"4ca05806-e559-459b-9e68-5f1ab17ce821","Type":"ContainerDied","Data":"237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c"} Jul 06 14:28:25.227852 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:25.227812 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="237b4609ffb904c93d36291c6a83045e3ba7c5c0cea37ff2cc0bce3dc5dc0e9c" Jul 06 14:28:25.246551 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:25.243707 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zz4bz" podStartSLOduration=2.658788998 podStartE2EDuration="5.243685157s" totalImagesPullingTime="2.584896159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:25.241083931 +0000 UTC m=+340.012106262" watchObservedRunningTime="2026-07-06 14:28:25.243685157 +0000 UTC m=+340.014707487" Jul 06 14:28:29.143260 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143227 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kzz4j"] Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143519 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="extract" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143535 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="extract" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143547 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="pull" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143562 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="pull" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143570 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="util" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143576 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="util" Jul 06 14:28:29.143667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.143636 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca05806-e559-459b-9e68-5f1ab17ce821" containerName="extract" Jul 06 14:28:29.148180 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.148163 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.150542 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.150514 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zk7zb\"" Jul 06 14:28:29.158413 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.158390 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kzz4j"] Jul 06 14:28:29.249615 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.249584 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-bound-sa-token\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.249794 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.249704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbqbk\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-kube-api-access-nbqbk\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.350223 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.350182 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbqbk\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-kube-api-access-nbqbk\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.350223 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.350225 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-bound-sa-token\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.358526 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.358497 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-bound-sa-token\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.358686 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.358667 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbqbk\" (UniqueName: \"kubernetes.io/projected/314f1cbd-7462-4040-9d39-72d41ff40345-kube-api-access-nbqbk\") pod \"cert-manager-55b4d954d5-kzz4j\" (UID: \"314f1cbd-7462-4040-9d39-72d41ff40345\") " pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.456909 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.456824 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-kzz4j" Jul 06 14:28:29.593414 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:29.593379 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-kzz4j"] Jul 06 14:28:29.596458 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:29.596422 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod314f1cbd_7462_4040_9d39_72d41ff40345.slice/crio-3f7ba20669d486a140dccd5f59160cc0e55d761ecc4ed4aa14c386c468ff5a39 WatchSource:0}: Error finding container 3f7ba20669d486a140dccd5f59160cc0e55d761ecc4ed4aa14c386c468ff5a39: Status 404 returned error can't find the container with id 3f7ba20669d486a140dccd5f59160cc0e55d761ecc4ed4aa14c386c468ff5a39 Jul 06 14:28:30.250522 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:30.250483 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kzz4j" event={"ID":"314f1cbd-7462-4040-9d39-72d41ff40345","Type":"ContainerStarted","Data":"38c2900d10566bcdcaf8ae402dcc1f03ebde826dc7cb609a0a84363303d76a64"} Jul 06 14:28:30.250915 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:30.250531 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-kzz4j" event={"ID":"314f1cbd-7462-4040-9d39-72d41ff40345","Type":"ContainerStarted","Data":"3f7ba20669d486a140dccd5f59160cc0e55d761ecc4ed4aa14c386c468ff5a39"} Jul 06 14:28:30.265663 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:30.265610 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-kzz4j" podStartSLOduration=1.265593146 podStartE2EDuration="1.265593146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:30.26418485 +0000 UTC m=+345.035207179" watchObservedRunningTime="2026-07-06 14:28:30.265593146 +0000 UTC m=+345.036615477" Jul 06 14:28:38.852220 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.852136 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh"] Jul 06 14:28:38.854509 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.854491 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:38.856427 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.856398 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d8xn2\"" Jul 06 14:28:38.856541 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.856468 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:28:38.856842 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.856822 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:28:38.865816 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:38.865787 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh"] Jul 06 14:28:39.026054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.026012 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgv6\" (UniqueName: \"kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.026054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.026058 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.026278 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.026087 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.127420 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.127339 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.127525 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.127440 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgv6\" (UniqueName: \"kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.127525 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.127480 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.127851 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.127825 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.127896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.127836 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.136930 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.136899 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgv6\" (UniqueName: \"kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.163868 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.163835 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:39.296250 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:39.296214 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh"] Jul 06 14:28:39.301005 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:39.300976 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4df436_e358_4552_bab2_a237037588eb.slice/crio-e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9 WatchSource:0}: Error finding container e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9: Status 404 returned error can't find the container with id e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9 Jul 06 14:28:40.294786 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:40.294752 2552 generic.go:356] "Generic (PLEG): container finished" podID="ca4df436-e358-4552-bab2-a237037588eb" containerID="03ffd4c22e0c9460fdedeb5e7483e58fba256e8e32129272f055e4b8c2a6048d" exitCode=0 Jul 06 14:28:40.295153 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:40.294806 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" event={"ID":"ca4df436-e358-4552-bab2-a237037588eb","Type":"ContainerDied","Data":"03ffd4c22e0c9460fdedeb5e7483e58fba256e8e32129272f055e4b8c2a6048d"} Jul 06 14:28:40.295153 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:40.294843 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" event={"ID":"ca4df436-e358-4552-bab2-a237037588eb","Type":"ContainerStarted","Data":"e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9"} Jul 06 14:28:41.300984 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:41.300946 2552 generic.go:356] "Generic (PLEG): container finished" podID="ca4df436-e358-4552-bab2-a237037588eb" containerID="2333f97f139483c8c13694a92f9a2cb100b932cc5325f1bdce53e057f7d296fd" exitCode=0 Jul 06 14:28:41.300984 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:41.300983 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" event={"ID":"ca4df436-e358-4552-bab2-a237037588eb","Type":"ContainerDied","Data":"2333f97f139483c8c13694a92f9a2cb100b932cc5325f1bdce53e057f7d296fd"} Jul 06 14:28:42.306998 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:42.306963 2552 generic.go:356] "Generic (PLEG): container finished" podID="ca4df436-e358-4552-bab2-a237037588eb" containerID="a1af4c79e9e84b7cc697bc658cec947953b01f34fbd4576ef9c1d29929677c04" exitCode=0 Jul 06 14:28:42.307457 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:42.307042 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" event={"ID":"ca4df436-e358-4552-bab2-a237037588eb","Type":"ContainerDied","Data":"a1af4c79e9e84b7cc697bc658cec947953b01f34fbd4576ef9c1d29929677c04"} Jul 06 14:28:43.444363 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.444307 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:43.563154 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.563049 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle\") pod \"ca4df436-e358-4552-bab2-a237037588eb\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " Jul 06 14:28:43.563154 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.563150 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlgv6\" (UniqueName: \"kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6\") pod \"ca4df436-e358-4552-bab2-a237037588eb\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " Jul 06 14:28:43.563427 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.563167 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util\") pod \"ca4df436-e358-4552-bab2-a237037588eb\" (UID: \"ca4df436-e358-4552-bab2-a237037588eb\") " Jul 06 14:28:43.563875 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.563838 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle" (OuterVolumeSpecName: "bundle") pod "ca4df436-e358-4552-bab2-a237037588eb" (UID: "ca4df436-e358-4552-bab2-a237037588eb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:43.565226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.565200 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6" (OuterVolumeSpecName: "kube-api-access-rlgv6") pod "ca4df436-e358-4552-bab2-a237037588eb" (UID: "ca4df436-e358-4552-bab2-a237037588eb"). InnerVolumeSpecName "kube-api-access-rlgv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:28:43.569192 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.569081 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util" (OuterVolumeSpecName: "util") pod "ca4df436-e358-4552-bab2-a237037588eb" (UID: "ca4df436-e358-4552-bab2-a237037588eb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:28:43.663943 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.663904 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlgv6\" (UniqueName: \"kubernetes.io/projected/ca4df436-e358-4552-bab2-a237037588eb-kube-api-access-rlgv6\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:43.663943 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.663938 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:43.663943 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:43.663950 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca4df436-e358-4552-bab2-a237037588eb-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:28:44.317809 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:44.317771 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" event={"ID":"ca4df436-e358-4552-bab2-a237037588eb","Type":"ContainerDied","Data":"e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9"} Jul 06 14:28:44.317809 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:44.317797 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5w2bh" Jul 06 14:28:44.318015 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:44.317821 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3ce48a3ea997fdbb7677df7d72972ab987680e6a4b2064e274c1665b68172c9" Jul 06 14:28:52.197949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.197906 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb"] Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198225 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="pull" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198238 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="pull" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198252 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="extract" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198257 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="extract" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198270 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="util" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198275 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="util" Jul 06 14:28:52.198355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.198339 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4df436-e358-4552-bab2-a237037588eb" containerName="extract" Jul 06 14:28:52.200735 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.200718 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.206405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.206378 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 14:28:52.206532 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.206384 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 14:28:52.206981 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.206962 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l564n\"" Jul 06 14:28:52.207302 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.207156 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 14:28:52.207636 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.207617 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 14:28:52.218309 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.218284 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb"] Jul 06 14:28:52.225981 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.225957 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-tmp\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.225981 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.225987 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-webhook-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.226158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.226007 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-manifests\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.226158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.226096 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9bvt\" (UniqueName: \"kubernetes.io/projected/36bf9105-a209-4b24-b346-85fe501c784e-kube-api-access-d9bvt\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.226158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.226141 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327069 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327030 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9bvt\" (UniqueName: \"kubernetes.io/projected/36bf9105-a209-4b24-b346-85fe501c784e-kube-api-access-d9bvt\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327069 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327068 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327277 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327240 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-tmp\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327367 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327274 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-webhook-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327367 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327301 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-manifests\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.327677 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.327653 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-manifests\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.329551 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.329530 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36bf9105-a209-4b24-b346-85fe501c784e-tmp\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.329705 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.329689 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-webhook-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.329790 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.329774 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36bf9105-a209-4b24-b346-85fe501c784e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.335447 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.335427 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9bvt\" (UniqueName: \"kubernetes.io/projected/36bf9105-a209-4b24-b346-85fe501c784e-kube-api-access-d9bvt\") pod \"opendatahub-operator-controller-manager-557ddcc9b5-vttbb\" (UID: \"36bf9105-a209-4b24-b346-85fe501c784e\") " pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.511004 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.510907 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:52.665725 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:52.665580 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb"] Jul 06 14:28:52.668716 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:28:52.668685 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36bf9105_a209_4b24_b346_85fe501c784e.slice/crio-b7433955efcd705a886366acf6025daf45e39094ff370ccdd3f72c5d84885196 WatchSource:0}: Error finding container b7433955efcd705a886366acf6025daf45e39094ff370ccdd3f72c5d84885196: Status 404 returned error can't find the container with id b7433955efcd705a886366acf6025daf45e39094ff370ccdd3f72c5d84885196 Jul 06 14:28:53.363782 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:53.363737 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" event={"ID":"36bf9105-a209-4b24-b346-85fe501c784e","Type":"ContainerStarted","Data":"b7433955efcd705a886366acf6025daf45e39094ff370ccdd3f72c5d84885196"} Jul 06 14:28:56.379819 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:56.379782 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" event={"ID":"36bf9105-a209-4b24-b346-85fe501c784e","Type":"ContainerStarted","Data":"236ddeef062dcbb61b7d5a7ea1220f626f05b05247bfa6116dcb8fbb372bd985"} Jul 06 14:28:57.385101 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:57.385070 2552 generic.go:356] "Generic (PLEG): container finished" podID="36bf9105-a209-4b24-b346-85fe501c784e" containerID="236ddeef062dcbb61b7d5a7ea1220f626f05b05247bfa6116dcb8fbb372bd985" exitCode=0 Jul 06 14:28:57.385515 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:57.385132 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" event={"ID":"36bf9105-a209-4b24-b346-85fe501c784e","Type":"ContainerDied","Data":"236ddeef062dcbb61b7d5a7ea1220f626f05b05247bfa6116dcb8fbb372bd985"} Jul 06 14:28:58.391015 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:58.390973 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" event={"ID":"36bf9105-a209-4b24-b346-85fe501c784e","Type":"ContainerStarted","Data":"2a7053f9bf0d8257d5b1f11611549cda16652ed3cd01c491b36a4ab8c5210f74"} Jul 06 14:28:58.391481 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:58.391099 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:28:58.414949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:28:58.414899 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" podStartSLOduration=1.7863469360000002 podStartE2EDuration="6.414883899s" totalImagesPullingTime="3.537093506s" totalInitContainerRuntime="1.091443457s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:58.41399328 +0000 UTC m=+373.185015630" watchObservedRunningTime="2026-07-06 14:28:58.414883899 +0000 UTC m=+373.185906229" Jul 06 14:29:09.397566 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:09.397529 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-557ddcc9b5-vttbb" Jul 06 14:29:11.781995 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.781957 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnznn"] Jul 06 14:29:11.785072 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.785053 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:11.788236 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.788215 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 14:29:11.788412 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.788389 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-98j8b\"" Jul 06 14:29:11.804187 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.804161 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnznn"] Jul 06 14:29:11.879955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.879921 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:11.880125 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.879969 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nxfb\" (UniqueName: \"kubernetes.io/projected/e8dfbec5-bf40-4b90-b72b-942679b69c68-kube-api-access-2nxfb\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:11.980949 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.980921 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nxfb\" (UniqueName: \"kubernetes.io/projected/e8dfbec5-bf40-4b90-b72b-942679b69c68-kube-api-access-2nxfb\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:11.981112 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.980989 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:11.981112 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:29:11.981092 2552 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 14:29:11.981203 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:29:11.981157 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert podName:e8dfbec5-bf40-4b90-b72b-942679b69c68 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:12.481139001 +0000 UTC m=+387.252161310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert") pod "odh-model-controller-7998c67dc4-gnznn" (UID: "e8dfbec5-bf40-4b90-b72b-942679b69c68") : secret "odh-model-controller-webhook-cert" not found Jul 06 14:29:11.998063 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:11.998036 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nxfb\" (UniqueName: \"kubernetes.io/projected/e8dfbec5-bf40-4b90-b72b-942679b69c68-kube-api-access-2nxfb\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:12.483641 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.483603 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:12.486244 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.486212 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8dfbec5-bf40-4b90-b72b-942679b69c68-cert\") pod \"odh-model-controller-7998c67dc4-gnznn\" (UID: \"e8dfbec5-bf40-4b90-b72b-942679b69c68\") " pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:12.592495 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.592453 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xk4x2"] Jul 06 14:29:12.595754 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.595731 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.598157 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.598130 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 14:29:12.632922 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.631312 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xk4x2"] Jul 06 14:29:12.685335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.685298 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxct\" (UniqueName: \"kubernetes.io/projected/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-kube-api-access-jvxct\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.685516 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.685360 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.694348 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.694287 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:12.786775 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.786744 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.787203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.786838 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxct\" (UniqueName: \"kubernetes.io/projected/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-kube-api-access-jvxct\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.787203 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:29:12.786928 2552 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 14:29:12.787203 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:29:12.787018 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs podName:666c79f7-e3be-4bbc-9cd5-13fe98bb2985 nodeName:}" failed. No retries permitted until 2026-07-06 14:29:13.286997194 +0000 UTC m=+388.058019505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs") pod "model-serving-api-54d89bd5fc-xk4x2" (UID: "666c79f7-e3be-4bbc-9cd5-13fe98bb2985") : secret "model-serving-api-tls" not found Jul 06 14:29:12.795603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.795568 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxct\" (UniqueName: \"kubernetes.io/projected/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-kube-api-access-jvxct\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:12.829473 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:12.829440 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gnznn"] Jul 06 14:29:12.832452 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:29:12.832423 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dfbec5_bf40_4b90_b72b_942679b69c68.slice/crio-b8e2c6a0227e7c2bdcb5a1eb0e53bf33fc7f8c91bfc3c4ecdd7a4db8da1f7bf1 WatchSource:0}: Error finding container b8e2c6a0227e7c2bdcb5a1eb0e53bf33fc7f8c91bfc3c4ecdd7a4db8da1f7bf1: Status 404 returned error can't find the container with id b8e2c6a0227e7c2bdcb5a1eb0e53bf33fc7f8c91bfc3c4ecdd7a4db8da1f7bf1 Jul 06 14:29:13.290795 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:13.290761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:13.293163 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:13.293141 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/666c79f7-e3be-4bbc-9cd5-13fe98bb2985-tls-certs\") pod \"model-serving-api-54d89bd5fc-xk4x2\" (UID: \"666c79f7-e3be-4bbc-9cd5-13fe98bb2985\") " pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:13.454093 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:13.454056 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" event={"ID":"e8dfbec5-bf40-4b90-b72b-942679b69c68","Type":"ContainerStarted","Data":"b8e2c6a0227e7c2bdcb5a1eb0e53bf33fc7f8c91bfc3c4ecdd7a4db8da1f7bf1"} Jul 06 14:29:13.506199 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:13.506166 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:13.666052 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:13.665959 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xk4x2"] Jul 06 14:29:13.668964 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:29:13.668928 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666c79f7_e3be_4bbc_9cd5_13fe98bb2985.slice/crio-96c76ccbd92750d31cca8a9a42605bc3b9a2514bf8251e5354dbe9cd9938fac7 WatchSource:0}: Error finding container 96c76ccbd92750d31cca8a9a42605bc3b9a2514bf8251e5354dbe9cd9938fac7: Status 404 returned error can't find the container with id 96c76ccbd92750d31cca8a9a42605bc3b9a2514bf8251e5354dbe9cd9938fac7 Jul 06 14:29:14.460603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.460560 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" event={"ID":"666c79f7-e3be-4bbc-9cd5-13fe98bb2985","Type":"ContainerStarted","Data":"96c76ccbd92750d31cca8a9a42605bc3b9a2514bf8251e5354dbe9cd9938fac7"} Jul 06 14:29:14.484470 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.484208 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr"] Jul 06 14:29:14.488177 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.488154 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.490483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.490440 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 14:29:14.490483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.490475 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 14:29:14.491528 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.490929 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-knrgv\"" Jul 06 14:29:14.491528 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.491137 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 14:29:14.491528 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.491333 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 14:29:14.498782 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.498720 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr"] Jul 06 14:29:14.603120 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.603081 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb74fabc-8555-437c-8959-7f50d3b3553e-tls-certs\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.603304 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.603143 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb74fabc-8555-437c-8959-7f50d3b3553e-tmp\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.603304 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.603282 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhp4r\" (UniqueName: \"kubernetes.io/projected/cb74fabc-8555-437c-8959-7f50d3b3553e-kube-api-access-lhp4r\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.704203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.704163 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhp4r\" (UniqueName: \"kubernetes.io/projected/cb74fabc-8555-437c-8959-7f50d3b3553e-kube-api-access-lhp4r\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.704203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.704211 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb74fabc-8555-437c-8959-7f50d3b3553e-tls-certs\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.704495 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.704248 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb74fabc-8555-437c-8959-7f50d3b3553e-tmp\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.706666 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.706635 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb74fabc-8555-437c-8959-7f50d3b3553e-tmp\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.706872 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.706853 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb74fabc-8555-437c-8959-7f50d3b3553e-tls-certs\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.720959 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.720892 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhp4r\" (UniqueName: \"kubernetes.io/projected/cb74fabc-8555-437c-8959-7f50d3b3553e-kube-api-access-lhp4r\") pod \"kube-auth-proxy-7b5f845fbc-bvglr\" (UID: \"cb74fabc-8555-437c-8959-7f50d3b3553e\") " pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.805843 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.805806 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" Jul 06 14:29:14.983847 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:14.983819 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr"] Jul 06 14:29:15.636507 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:29:15.636476 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb74fabc_8555_437c_8959_7f50d3b3553e.slice/crio-e1da903f3e8a1e40aafc6c21438f0d567925b39ea835c35944d8ce4139194a10 WatchSource:0}: Error finding container e1da903f3e8a1e40aafc6c21438f0d567925b39ea835c35944d8ce4139194a10: Status 404 returned error can't find the container with id e1da903f3e8a1e40aafc6c21438f0d567925b39ea835c35944d8ce4139194a10 Jul 06 14:29:15.758594 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.758561 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:15.762236 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.762211 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.764346 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.764305 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 14:29:15.764492 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.764466 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 14:29:15.764631 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.764309 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 14:29:15.764631 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.764338 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qbkzq\"" Jul 06 14:29:15.764772 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.764665 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 14:29:15.775964 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.775931 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:15.814404 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814373 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814404 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814425 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48rk\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814465 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814495 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814513 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814535 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.814661 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.814555 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915648 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915723 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48rk\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915762 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915793 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915819 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.915976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.915864 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.916618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.916566 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.918579 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.918555 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.918767 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.918744 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.918998 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.918968 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.919126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.919099 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.924350 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.924282 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48rk\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:15.924471 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:15.924378 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token\") pod \"istiod-openshift-gateway-7574459656-4wsxl\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:16.074601 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:16.074562 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:16.478262 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:16.478217 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" event={"ID":"cb74fabc-8555-437c-8959-7f50d3b3553e","Type":"ContainerStarted","Data":"e1da903f3e8a1e40aafc6c21438f0d567925b39ea835c35944d8ce4139194a10"} Jul 06 14:29:16.712842 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:16.712766 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:16.715093 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:29:16.715062 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61005db4_d9a3_41eb_9796_0d19500351cb.slice/crio-64777e7702554b8f85c3c7ff1e0f1a9efda541a455bda16a220dde01033010ae WatchSource:0}: Error finding container 64777e7702554b8f85c3c7ff1e0f1a9efda541a455bda16a220dde01033010ae: Status 404 returned error can't find the container with id 64777e7702554b8f85c3c7ff1e0f1a9efda541a455bda16a220dde01033010ae Jul 06 14:29:17.485092 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.485046 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" event={"ID":"666c79f7-e3be-4bbc-9cd5-13fe98bb2985","Type":"ContainerStarted","Data":"fff5fa54e05a00a87cbf5e909e89dc64d7f629129ba99130988cd8dcda17396e"} Jul 06 14:29:17.485290 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.485162 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:17.486695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.486670 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" event={"ID":"e8dfbec5-bf40-4b90-b72b-942679b69c68","Type":"ContainerStarted","Data":"c86f837d363fb77a1cc54b527c01e4d452528df025049dd9d770e22ec3d84f54"} Jul 06 14:29:17.486812 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.486723 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:17.487862 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.487835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" event={"ID":"61005db4-d9a3-41eb-9796-0d19500351cb","Type":"ContainerStarted","Data":"64777e7702554b8f85c3c7ff1e0f1a9efda541a455bda16a220dde01033010ae"} Jul 06 14:29:17.504039 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.503978 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" podStartSLOduration=2.6053975879999998 podStartE2EDuration="5.503956603s" totalImagesPullingTime="2.898559015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:17.50208022 +0000 UTC m=+392.273102556" watchObservedRunningTime="2026-07-06 14:29:17.503956603 +0000 UTC m=+392.274978935" Jul 06 14:29:17.522215 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:17.522165 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" podStartSLOduration=2.784490757 podStartE2EDuration="6.522149951s" totalImagesPullingTime="3.737659194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:17.52063585 +0000 UTC m=+392.291658197" watchObservedRunningTime="2026-07-06 14:29:17.522149951 +0000 UTC m=+392.293172280" Jul 06 14:29:19.536509 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:19.536458 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:29:19.536830 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:19.536558 2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:29:20.389309 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.389271 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z"] Jul 06 14:29:20.392724 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.392698 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.396209 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396186 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 14:29:20.396382 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396188 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 14:29:20.396382 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396187 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7l925\"" Jul 06 14:29:20.396477 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396438 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:29:20.396723 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396700 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 14:29:20.396834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.396738 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 14:29:20.404755 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.404734 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z"] Jul 06 14:29:20.459620 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.459577 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.459620 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.459622 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5qs\" (UniqueName: \"kubernetes.io/projected/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-kube-api-access-qd5qs\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.459828 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.459654 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-metrics-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.459828 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.459749 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-manager-config\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.510345 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.510279 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" event={"ID":"61005db4-d9a3-41eb-9796-0d19500351cb","Type":"ContainerStarted","Data":"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958"} Jul 06 14:29:20.510540 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.510436 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:20.512146 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.512111 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" event={"ID":"cb74fabc-8555-437c-8959-7f50d3b3553e","Type":"ContainerStarted","Data":"73de3384f9849b42a5f23909e6cf614031b1d316ca239fa51d719a325b460ed5"} Jul 06 14:29:20.512635 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.512607 2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4wsxl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:20.512757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.512659 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:20.560369 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.560310 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.560369 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.560373 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5qs\" (UniqueName: \"kubernetes.io/projected/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-kube-api-access-qd5qs\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.560887 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.560398 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-metrics-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.560887 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.560520 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-manager-config\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.561405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.561383 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-manager-config\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.563282 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.563259 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-metrics-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.563472 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.563450 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-cert\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.574423 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.574396 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5qs\" (UniqueName: \"kubernetes.io/projected/ffedf392-55a8-4d69-9a47-d6afdedcb8c8-kube-api-access-qd5qs\") pod \"lws-controller-manager-66c4cbf4f4-glf8z\" (UID: \"ffedf392-55a8-4d69-9a47-d6afdedcb8c8\") " pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.668989 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.668885 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" podStartSLOduration=2.850702967 podStartE2EDuration="5.668867758s" totalImagesPullingTime="2.818164791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:20.665405191 +0000 UTC m=+395.436427514" watchObservedRunningTime="2026-07-06 14:29:20.668867758 +0000 UTC m=+395.439890088" Jul 06 14:29:20.703861 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.703828 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:20.732410 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.732353 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b5f845fbc-bvglr" podStartSLOduration=2.895453232 podStartE2EDuration="6.732316851s" totalImagesPullingTime="3.836863619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:20.728849131 +0000 UTC m=+395.499871497" watchObservedRunningTime="2026-07-06 14:29:20.732316851 +0000 UTC m=+395.503339180" Jul 06 14:29:20.877769 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:20.877733 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z"] Jul 06 14:29:20.881060 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:29:20.881035 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffedf392_55a8_4d69_9a47_d6afdedcb8c8.slice/crio-f05a6eec2029689062db3d16ae73270d44e20465027d87564f2232a8e69d33a8 WatchSource:0}: Error finding container f05a6eec2029689062db3d16ae73270d44e20465027d87564f2232a8e69d33a8: Status 404 returned error can't find the container with id f05a6eec2029689062db3d16ae73270d44e20465027d87564f2232a8e69d33a8 Jul 06 14:29:21.517778 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:21.517732 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" event={"ID":"ffedf392-55a8-4d69-9a47-d6afdedcb8c8","Type":"ContainerStarted","Data":"f05a6eec2029689062db3d16ae73270d44e20465027d87564f2232a8e69d33a8"} Jul 06 14:29:21.518529 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:21.518504 2552 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4wsxl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:29:21.518623 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:21.518549 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:29:23.532700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:23.532661 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" event={"ID":"ffedf392-55a8-4d69-9a47-d6afdedcb8c8","Type":"ContainerStarted","Data":"939a1e64bf707ca5d140125f1729a0d15fbcc838d4b90f731e23bac942a1a095"} Jul 06 14:29:23.533171 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:23.532901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:29:23.626015 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:23.625961 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" podStartSLOduration=2.043705433 podStartE2EDuration="3.62594398s" totalImagesPullingTime="1.582238547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:23.623455949 +0000 UTC m=+398.394478280" watchObservedRunningTime="2026-07-06 14:29:23.62594398 +0000 UTC m=+398.396966309" Jul 06 14:29:24.280907 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.280869 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:24.281138 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.281086 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" containerID="cri-o://e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958" gracePeriod=30 Jul 06 14:29:24.281859 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.281765 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:24.535028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.535000 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:24.538700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.538674 2552 generic.go:356] "Generic (PLEG): container finished" podID="61005db4-d9a3-41eb-9796-0d19500351cb" containerID="e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958" exitCode=0 Jul 06 14:29:24.538830 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.538713 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" Jul 06 14:29:24.538830 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.538752 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" event={"ID":"61005db4-d9a3-41eb-9796-0d19500351cb","Type":"ContainerDied","Data":"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958"} Jul 06 14:29:24.538830 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.538806 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl" event={"ID":"61005db4-d9a3-41eb-9796-0d19500351cb","Type":"ContainerDied","Data":"64777e7702554b8f85c3c7ff1e0f1a9efda541a455bda16a220dde01033010ae"} Jul 06 14:29:24.539000 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.538832 2552 scope.go:117] "RemoveContainer" containerID="e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958" Jul 06 14:29:24.547251 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.547233 2552 scope.go:117] "RemoveContainer" containerID="e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958" Jul 06 14:29:24.547534 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:29:24.547515 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958\": container with ID starting with e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958 not found: ID does not exist" containerID="e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958" Jul 06 14:29:24.547603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.547554 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958"} err="failed to get container status \"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958\": rpc error: code = NotFound desc = could not find container \"e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958\": container with ID starting with e4910b3822d569188af9e21bb24c1c4123a5cbb0f2274313ee6f91069e70f958 not found: ID does not exist" Jul 06 14:29:24.596052 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596025 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596217 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596075 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596217 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596103 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p48rk\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596217 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596143 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596217 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596188 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596471 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596229 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596471 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596277 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert\") pod \"61005db4-d9a3-41eb-9796-0d19500351cb\" (UID: \"61005db4-d9a3-41eb-9796-0d19500351cb\") " Jul 06 14:29:24.596569 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.596478 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:29:24.598757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.598715 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:29:24.598757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.598721 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:24.598932 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.598831 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs" (OuterVolumeSpecName: "local-certs") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:29:24.598932 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.598917 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts" (OuterVolumeSpecName: "cacerts") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:24.599012 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.598976 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk" (OuterVolumeSpecName: "kube-api-access-p48rk") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "kube-api-access-p48rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:29:24.599120 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.599100 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "61005db4-d9a3-41eb-9796-0d19500351cb" (UID: "61005db4-d9a3-41eb-9796-0d19500351cb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:24.697483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697444 2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-istio-token\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697476 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p48rk\" (UniqueName: \"kubernetes.io/projected/61005db4-d9a3-41eb-9796-0d19500351cb-kube-api-access-p48rk\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697700 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697566 2552 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-cacerts\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697748 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697698 2552 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-kubeconfig\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697748 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697723 2552 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/61005db4-d9a3-41eb-9796-0d19500351cb-local-certs\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697748 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697739 2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-dns-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.697838 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.697748 2552 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/61005db4-d9a3-41eb-9796-0d19500351cb-istio-csr-ca-configmap\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:29:24.861466 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.861431 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:24.866335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:24.866304 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4wsxl"] Jul 06 14:29:25.756053 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:25.756023 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" path="/var/lib/kubelet/pods/61005db4-d9a3-41eb-9796-0d19500351cb/volumes" Jul 06 14:29:28.497896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:28.497858 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gnznn" Jul 06 14:29:28.499723 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:28.499707 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xk4x2" Jul 06 14:29:34.541485 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:29:34.541448 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66c4cbf4f4-glf8z" Jul 06 14:30:28.320367 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.320314 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv"] Jul 06 14:30:28.320783 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.320628 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" Jul 06 14:30:28.320783 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.320640 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" Jul 06 14:30:28.320783 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.320695 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="61005db4-d9a3-41eb-9796-0d19500351cb" containerName="discovery" Jul 06 14:30:28.323056 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.323038 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.324942 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.324918 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:30:28.325397 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.325375 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lkvjk\"" Jul 06 14:30:28.325488 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.325446 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:30:28.332065 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.332041 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv"] Jul 06 14:30:28.416028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.415994 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.416028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.416036 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.416265 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.416078 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzwd\" (UniqueName: \"kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.517608 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.517573 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzwd\" (UniqueName: \"kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.517787 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.517627 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.517787 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.517657 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.517981 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.517966 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.518028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.518008 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.525708 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.525681 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzwd\" (UniqueName: \"kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.632879 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.632786 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:28.763226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.763194 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv"] Jul 06 14:30:28.765778 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:30:28.765753 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cb9813_4429_4377_821a_232f74f758ad.slice/crio-478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472 WatchSource:0}: Error finding container 478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472: Status 404 returned error can't find the container with id 478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472 Jul 06 14:30:28.816620 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:28.816588 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" event={"ID":"11cb9813-4429-4377-821a-232f74f758ad","Type":"ContainerStarted","Data":"478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472"} Jul 06 14:30:29.152696 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.152662 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4"] Jul 06 14:30:29.154955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.154930 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.163965 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.163937 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4"] Jul 06 14:30:29.224440 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.224404 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5dmb\" (UniqueName: \"kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.224610 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.224483 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.224610 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.224508 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.325838 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.325803 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.326294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.325848 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5dmb\" (UniqueName: \"kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.326294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.325905 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.326294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.326278 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.326294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.326279 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.333483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.333457 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5dmb\" (UniqueName: \"kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.464354 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.464254 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:29.516084 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.516052 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8"] Jul 06 14:30:29.519880 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.519853 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.530982 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.530946 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8"] Jul 06 14:30:29.617028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.617001 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4"] Jul 06 14:30:29.619372 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:30:29.619336 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851941b8_0c97_47eb_8011_907044958f5f.slice/crio-e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802 WatchSource:0}: Error finding container e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802: Status 404 returned error can't find the container with id e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802 Jul 06 14:30:29.631561 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.631522 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.631689 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.631580 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.631689 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.631639 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccqf2\" (UniqueName: \"kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.733118 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.732927 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.733118 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.732961 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.733118 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.732987 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccqf2\" (UniqueName: \"kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.733397 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.733372 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.733482 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.733416 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.742179 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.742155 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccqf2\" (UniqueName: \"kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.823444 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.823409 2552 generic.go:356] "Generic (PLEG): container finished" podID="11cb9813-4429-4377-821a-232f74f758ad" containerID="124a498eba041be0c2524394552567fde55a8c51c8738b9dbc0fed125d8a53aa" exitCode=0 Jul 06 14:30:29.823635 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.823495 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" event={"ID":"11cb9813-4429-4377-821a-232f74f758ad","Type":"ContainerDied","Data":"124a498eba041be0c2524394552567fde55a8c51c8738b9dbc0fed125d8a53aa"} Jul 06 14:30:29.824940 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.824912 2552 generic.go:356] "Generic (PLEG): container finished" podID="851941b8-0c97-47eb-8011-907044958f5f" containerID="8184fc4c279d7f0608a38670ae98be5b477c1b399e60633e2c3459ad63344bfc" exitCode=0 Jul 06 14:30:29.825054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.824969 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" event={"ID":"851941b8-0c97-47eb-8011-907044958f5f","Type":"ContainerDied","Data":"8184fc4c279d7f0608a38670ae98be5b477c1b399e60633e2c3459ad63344bfc"} Jul 06 14:30:29.825054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.825013 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" event={"ID":"851941b8-0c97-47eb-8011-907044958f5f","Type":"ContainerStarted","Data":"e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802"} Jul 06 14:30:29.855521 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.855489 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:29.921638 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.921605 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc"] Jul 06 14:30:29.926298 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.926278 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:29.933399 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.933315 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc"] Jul 06 14:30:29.991663 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:29.991585 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8"] Jul 06 14:30:29.994530 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:30:29.994501 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764666f5_88ad_4486_841e_4e6bed0c2692.slice/crio-4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384 WatchSource:0}: Error finding container 4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384: Status 404 returned error can't find the container with id 4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384 Jul 06 14:30:30.034824 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.034801 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.034945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.034843 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l5dh\" (UniqueName: \"kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.034945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.034901 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.135995 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.135959 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.136168 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.136002 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l5dh\" (UniqueName: \"kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.136168 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.136111 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.136335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.136304 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.136450 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.136431 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.145315 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.145294 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l5dh\" (UniqueName: \"kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.237743 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.237712 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:30.375643 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.375606 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc"] Jul 06 14:30:30.384657 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:30:30.384620 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bba9bba_b551_49d1_8802_b407209cd504.slice/crio-2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0 WatchSource:0}: Error finding container 2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0: Status 404 returned error can't find the container with id 2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0 Jul 06 14:30:30.830897 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.830860 2552 generic.go:356] "Generic (PLEG): container finished" podID="1bba9bba-b551-49d1-8802-b407209cd504" containerID="b07ca2c62998ef145b646e6ed07d26daca1869a7864cdb8c501fd9ae3611e5c0" exitCode=0 Jul 06 14:30:30.831024 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.830899 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" event={"ID":"1bba9bba-b551-49d1-8802-b407209cd504","Type":"ContainerDied","Data":"b07ca2c62998ef145b646e6ed07d26daca1869a7864cdb8c501fd9ae3611e5c0"} Jul 06 14:30:30.831024 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.830949 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" event={"ID":"1bba9bba-b551-49d1-8802-b407209cd504","Type":"ContainerStarted","Data":"2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0"} Jul 06 14:30:30.832673 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.832650 2552 generic.go:356] "Generic (PLEG): container finished" podID="764666f5-88ad-4486-841e-4e6bed0c2692" containerID="5fe6192a757e62160c0a8bc5aa8454fc28c54822a21379c9b7f77cb1d29207c9" exitCode=0 Jul 06 14:30:30.832774 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.832744 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" event={"ID":"764666f5-88ad-4486-841e-4e6bed0c2692","Type":"ContainerDied","Data":"5fe6192a757e62160c0a8bc5aa8454fc28c54822a21379c9b7f77cb1d29207c9"} Jul 06 14:30:30.832835 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.832796 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" event={"ID":"764666f5-88ad-4486-841e-4e6bed0c2692","Type":"ContainerStarted","Data":"4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384"} Jul 06 14:30:30.834899 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.834881 2552 generic.go:356] "Generic (PLEG): container finished" podID="11cb9813-4429-4377-821a-232f74f758ad" containerID="6a96b6d8d1081d35c991c897da66aa7be797664565087ad6473d0d43ca2468b9" exitCode=0 Jul 06 14:30:30.834992 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.834909 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" event={"ID":"11cb9813-4429-4377-821a-232f74f758ad","Type":"ContainerDied","Data":"6a96b6d8d1081d35c991c897da66aa7be797664565087ad6473d0d43ca2468b9"} Jul 06 14:30:30.837166 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.837132 2552 generic.go:356] "Generic (PLEG): container finished" podID="851941b8-0c97-47eb-8011-907044958f5f" containerID="3eb584fbfa62e6f766e2d64795bd817640165743ab367124b96c121329c7603e" exitCode=0 Jul 06 14:30:30.837261 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:30.837212 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" event={"ID":"851941b8-0c97-47eb-8011-907044958f5f","Type":"ContainerDied","Data":"3eb584fbfa62e6f766e2d64795bd817640165743ab367124b96c121329c7603e"} Jul 06 14:30:31.844494 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.844460 2552 generic.go:356] "Generic (PLEG): container finished" podID="11cb9813-4429-4377-821a-232f74f758ad" containerID="a47cf40a1fa2205f0bf347b0cda22e7625b3dec3962f0ad439504d47b0efdee0" exitCode=0 Jul 06 14:30:31.844911 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.844550 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" event={"ID":"11cb9813-4429-4377-821a-232f74f758ad","Type":"ContainerDied","Data":"a47cf40a1fa2205f0bf347b0cda22e7625b3dec3962f0ad439504d47b0efdee0"} Jul 06 14:30:31.846618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.846594 2552 generic.go:356] "Generic (PLEG): container finished" podID="851941b8-0c97-47eb-8011-907044958f5f" containerID="0e079d2f0ab4280bc51dc6d83c9577d701533157c056d0439b7dfd2f57fd1bb5" exitCode=0 Jul 06 14:30:31.846740 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.846682 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" event={"ID":"851941b8-0c97-47eb-8011-907044958f5f","Type":"ContainerDied","Data":"0e079d2f0ab4280bc51dc6d83c9577d701533157c056d0439b7dfd2f57fd1bb5"} Jul 06 14:30:31.848634 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.848611 2552 generic.go:356] "Generic (PLEG): container finished" podID="1bba9bba-b551-49d1-8802-b407209cd504" containerID="7846d8da337aa22896057ed3f21260851fd000067439bb1796e4b29d3061cfc2" exitCode=0 Jul 06 14:30:31.848740 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.848692 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" event={"ID":"1bba9bba-b551-49d1-8802-b407209cd504","Type":"ContainerDied","Data":"7846d8da337aa22896057ed3f21260851fd000067439bb1796e4b29d3061cfc2"} Jul 06 14:30:31.850878 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.850856 2552 generic.go:356] "Generic (PLEG): container finished" podID="764666f5-88ad-4486-841e-4e6bed0c2692" containerID="32d1be0b58b55d8d45cf90eea72143198b636e961c8a258531ec44ebdebcb565" exitCode=0 Jul 06 14:30:31.850965 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:31.850930 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" event={"ID":"764666f5-88ad-4486-841e-4e6bed0c2692","Type":"ContainerDied","Data":"32d1be0b58b55d8d45cf90eea72143198b636e961c8a258531ec44ebdebcb565"} Jul 06 14:30:32.857732 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:32.857648 2552 generic.go:356] "Generic (PLEG): container finished" podID="764666f5-88ad-4486-841e-4e6bed0c2692" containerID="cf5ed430b1cc12a4136e33cce5e41d18e02837c300c875cfb6bd59d1583d68c2" exitCode=0 Jul 06 14:30:32.858133 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:32.857748 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" event={"ID":"764666f5-88ad-4486-841e-4e6bed0c2692","Type":"ContainerDied","Data":"cf5ed430b1cc12a4136e33cce5e41d18e02837c300c875cfb6bd59d1583d68c2"} Jul 06 14:30:32.859803 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:32.859777 2552 generic.go:356] "Generic (PLEG): container finished" podID="1bba9bba-b551-49d1-8802-b407209cd504" containerID="9f7a70ee96c80c328f3b796c7daea1d878a1ce81dedda523e134cbab4c611b2f" exitCode=0 Jul 06 14:30:32.859926 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:32.859860 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" event={"ID":"1bba9bba-b551-49d1-8802-b407209cd504","Type":"ContainerDied","Data":"9f7a70ee96c80c328f3b796c7daea1d878a1ce81dedda523e134cbab4c611b2f"} Jul 06 14:30:33.009744 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.009720 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:33.037350 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.037300 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:33.171994 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.171898 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5dmb\" (UniqueName: \"kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb\") pod \"851941b8-0c97-47eb-8011-907044958f5f\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " Jul 06 14:30:33.171994 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.171951 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util\") pod \"851941b8-0c97-47eb-8011-907044958f5f\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " Jul 06 14:30:33.171994 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.171982 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle\") pod \"851941b8-0c97-47eb-8011-907044958f5f\" (UID: \"851941b8-0c97-47eb-8011-907044958f5f\") " Jul 06 14:30:33.172244 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.172016 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdzwd\" (UniqueName: \"kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd\") pod \"11cb9813-4429-4377-821a-232f74f758ad\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " Jul 06 14:30:33.172244 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.172051 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util\") pod \"11cb9813-4429-4377-821a-232f74f758ad\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " Jul 06 14:30:33.172244 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.172142 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle\") pod \"11cb9813-4429-4377-821a-232f74f758ad\" (UID: \"11cb9813-4429-4377-821a-232f74f758ad\") " Jul 06 14:30:33.172725 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.172693 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle" (OuterVolumeSpecName: "bundle") pod "851941b8-0c97-47eb-8011-907044958f5f" (UID: "851941b8-0c97-47eb-8011-907044958f5f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:33.172916 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.172882 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle" (OuterVolumeSpecName: "bundle") pod "11cb9813-4429-4377-821a-232f74f758ad" (UID: "11cb9813-4429-4377-821a-232f74f758ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:33.174819 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.174789 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd" (OuterVolumeSpecName: "kube-api-access-zdzwd") pod "11cb9813-4429-4377-821a-232f74f758ad" (UID: "11cb9813-4429-4377-821a-232f74f758ad"). InnerVolumeSpecName "kube-api-access-zdzwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:33.174901 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.174825 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb" (OuterVolumeSpecName: "kube-api-access-p5dmb") pod "851941b8-0c97-47eb-8011-907044958f5f" (UID: "851941b8-0c97-47eb-8011-907044958f5f"). InnerVolumeSpecName "kube-api-access-p5dmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:33.177448 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.177429 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util" (OuterVolumeSpecName: "util") pod "851941b8-0c97-47eb-8011-907044958f5f" (UID: "851941b8-0c97-47eb-8011-907044958f5f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:33.178088 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.178065 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util" (OuterVolumeSpecName: "util") pod "11cb9813-4429-4377-821a-232f74f758ad" (UID: "11cb9813-4429-4377-821a-232f74f758ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:33.273246 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273210 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5dmb\" (UniqueName: \"kubernetes.io/projected/851941b8-0c97-47eb-8011-907044958f5f-kube-api-access-p5dmb\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.273246 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273240 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.273246 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273250 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/851941b8-0c97-47eb-8011-907044958f5f-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.273246 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273259 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdzwd\" (UniqueName: \"kubernetes.io/projected/11cb9813-4429-4377-821a-232f74f758ad-kube-api-access-zdzwd\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.273524 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273267 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.273524 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.273275 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11cb9813-4429-4377-821a-232f74f758ad-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:33.866938 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.866860 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" Jul 06 14:30:33.866938 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.866852 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv" event={"ID":"11cb9813-4429-4377-821a-232f74f758ad","Type":"ContainerDied","Data":"478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472"} Jul 06 14:30:33.867392 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.866971 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="478cb46f8f741ca8c073496e9a877e5f9c8dfcc1a83eae4f4788af156927d472" Jul 06 14:30:33.868834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.868793 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" event={"ID":"851941b8-0c97-47eb-8011-907044958f5f","Type":"ContainerDied","Data":"e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802"} Jul 06 14:30:33.868976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.868862 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e34f173fa201c0a4b93a775ff0acb05564aa6bf072b878205187826452588802" Jul 06 14:30:33.869554 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:33.869153 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4" Jul 06 14:30:34.017824 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.017801 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:34.020412 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.020390 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:34.182649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182550 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle\") pod \"1bba9bba-b551-49d1-8802-b407209cd504\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " Jul 06 14:30:34.182649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182589 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util\") pod \"1bba9bba-b551-49d1-8802-b407209cd504\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " Jul 06 14:30:34.182649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182611 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle\") pod \"764666f5-88ad-4486-841e-4e6bed0c2692\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " Jul 06 14:30:34.182649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182629 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l5dh\" (UniqueName: \"kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh\") pod \"1bba9bba-b551-49d1-8802-b407209cd504\" (UID: \"1bba9bba-b551-49d1-8802-b407209cd504\") " Jul 06 14:30:34.182955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182680 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccqf2\" (UniqueName: \"kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2\") pod \"764666f5-88ad-4486-841e-4e6bed0c2692\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " Jul 06 14:30:34.182955 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.182743 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util\") pod \"764666f5-88ad-4486-841e-4e6bed0c2692\" (UID: \"764666f5-88ad-4486-841e-4e6bed0c2692\") " Jul 06 14:30:34.183268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.183235 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle" (OuterVolumeSpecName: "bundle") pod "764666f5-88ad-4486-841e-4e6bed0c2692" (UID: "764666f5-88ad-4486-841e-4e6bed0c2692"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:34.183268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.183239 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle" (OuterVolumeSpecName: "bundle") pod "1bba9bba-b551-49d1-8802-b407209cd504" (UID: "1bba9bba-b551-49d1-8802-b407209cd504"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:34.184970 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.184941 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh" (OuterVolumeSpecName: "kube-api-access-2l5dh") pod "1bba9bba-b551-49d1-8802-b407209cd504" (UID: "1bba9bba-b551-49d1-8802-b407209cd504"). InnerVolumeSpecName "kube-api-access-2l5dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:34.185050 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.184943 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2" (OuterVolumeSpecName: "kube-api-access-ccqf2") pod "764666f5-88ad-4486-841e-4e6bed0c2692" (UID: "764666f5-88ad-4486-841e-4e6bed0c2692"). InnerVolumeSpecName "kube-api-access-ccqf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:34.188068 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.188037 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util" (OuterVolumeSpecName: "util") pod "1bba9bba-b551-49d1-8802-b407209cd504" (UID: "1bba9bba-b551-49d1-8802-b407209cd504"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:34.188737 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.188716 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util" (OuterVolumeSpecName: "util") pod "764666f5-88ad-4486-841e-4e6bed0c2692" (UID: "764666f5-88ad-4486-841e-4e6bed0c2692"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:30:34.284098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284046 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.284098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284090 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.284098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284100 2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bba9bba-b551-49d1-8802-b407209cd504-util\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.284098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284108 2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/764666f5-88ad-4486-841e-4e6bed0c2692-bundle\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.284098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284116 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l5dh\" (UniqueName: \"kubernetes.io/projected/1bba9bba-b551-49d1-8802-b407209cd504-kube-api-access-2l5dh\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.284431 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.284126 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccqf2\" (UniqueName: \"kubernetes.io/projected/764666f5-88ad-4486-841e-4e6bed0c2692-kube-api-access-ccqf2\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:30:34.874850 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.874820 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" Jul 06 14:30:34.874850 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.874834 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc" event={"ID":"1bba9bba-b551-49d1-8802-b407209cd504","Type":"ContainerDied","Data":"2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0"} Jul 06 14:30:34.875418 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.874886 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2003c36d58e5c56437ed0bf6c949fffb5f9924595c75f622929acc3ee991bfb0" Jul 06 14:30:34.876608 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.876593 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" Jul 06 14:30:34.876608 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.876596 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8" event={"ID":"764666f5-88ad-4486-841e-4e6bed0c2692","Type":"ContainerDied","Data":"4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384"} Jul 06 14:30:34.876744 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:34.876630 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bc278d063d93277d44d0db7ea9458d1781696ea6dcc12742bce80962f162384" Jul 06 14:30:55.728075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728036 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hg278"] Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728367 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728379 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728388 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728393 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728400 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728406 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728411 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="extract" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728416 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="extract" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728424 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728429 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728438 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728443 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="util" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728451 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728458 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728464 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="extract" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728469 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="extract" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728474 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="pull" Jul 06 14:30:55.728468 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728479 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="pull" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728486 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728490 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728497 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="util" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728502 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="util" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728508 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728513 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728565 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bba9bba-b551-49d1-8802-b407209cd504" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728576 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="851941b8-0c97-47eb-8011-907044958f5f" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728582 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cb9813-4429-4377-821a-232f74f758ad" containerName="extract" Jul 06 14:30:55.728993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.728587 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="764666f5-88ad-4486-841e-4e6bed0c2692" containerName="extract" Jul 06 14:30:55.731409 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.731392 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.734243 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.734217 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:30:55.734393 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.734251 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:30:55.734393 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.734336 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 14:30:55.734393 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.734375 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lkvjk\"" Jul 06 14:30:55.734781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.734765 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 14:30:55.742409 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.742378 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hg278"] Jul 06 14:30:55.757018 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.756983 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6da2126a-d27a-454e-82ec-7f64e3615fe3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.757220 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.757133 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6da2126a-d27a-454e-82ec-7f64e3615fe3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.757220 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.757165 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkssk\" (UniqueName: \"kubernetes.io/projected/6da2126a-d27a-454e-82ec-7f64e3615fe3-kube-api-access-fkssk\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.858209 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.858162 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6da2126a-d27a-454e-82ec-7f64e3615fe3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.858455 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.858243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6da2126a-d27a-454e-82ec-7f64e3615fe3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.858455 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.858263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkssk\" (UniqueName: \"kubernetes.io/projected/6da2126a-d27a-454e-82ec-7f64e3615fe3-kube-api-access-fkssk\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.858919 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.858894 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6da2126a-d27a-454e-82ec-7f64e3615fe3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.860752 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.860720 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6da2126a-d27a-454e-82ec-7f64e3615fe3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:55.866086 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:55.866059 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkssk\" (UniqueName: \"kubernetes.io/projected/6da2126a-d27a-454e-82ec-7f64e3615fe3-kube-api-access-fkssk\") pod \"kuadrant-console-plugin-66b5774985-hg278\" (UID: \"6da2126a-d27a-454e-82ec-7f64e3615fe3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:56.041264 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:56.041230 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" Jul 06 14:30:56.181538 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:56.181501 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hg278"] Jul 06 14:30:56.185369 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:30:56.185333 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6da2126a_d27a_454e_82ec_7f64e3615fe3.slice/crio-654ae9838b869774edac14a704c976bec7608894526c0a8ed2adcb0ff9d9056a WatchSource:0}: Error finding container 654ae9838b869774edac14a704c976bec7608894526c0a8ed2adcb0ff9d9056a: Status 404 returned error can't find the container with id 654ae9838b869774edac14a704c976bec7608894526c0a8ed2adcb0ff9d9056a Jul 06 14:30:56.985003 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:30:56.984965 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" event={"ID":"6da2126a-d27a-454e-82ec-7f64e3615fe3","Type":"ContainerStarted","Data":"654ae9838b869774edac14a704c976bec7608894526c0a8ed2adcb0ff9d9056a"} Jul 06 14:31:20.117538 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:20.117497 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" event={"ID":"6da2126a-d27a-454e-82ec-7f64e3615fe3","Type":"ContainerStarted","Data":"f8105055df915b5e3a7ae6b44c7bf40136f7625ec990e8479a766d528b3e669a"} Jul 06 14:31:20.135665 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:20.135612 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hg278" podStartSLOduration=1.791794758 podStartE2EDuration="25.135596989s" totalImagesPullingTime="23.343802231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:20.133507782 +0000 UTC m=+514.904530111" watchObservedRunningTime="2026-07-06 14:31:20.135596989 +0000 UTC m=+514.906619319" Jul 06 14:31:44.717075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:44.716990 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:45.326222 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.326185 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:45.326222 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.326220 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:31:45.326444 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.326333 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:45.328397 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.328375 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sfc8j\"" Jul 06 14:31:45.340523 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.340497 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:31:45.340635 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.340586 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:31:45.382229 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.382198 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5hrd\" (UniqueName: \"kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd\") pod \"authorino-75d556bd77-mk8c6\" (UID: \"b807bb3c-d204-4ca2-8eb3-376fc6a83344\") " pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:31:45.482825 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.482790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn2m6\" (UniqueName: \"kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6\") pod \"authorino-58cd955675-tngml\" (UID: \"8239aae2-b569-4619-9cb8-4478eef7fa7b\") " pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:45.483000 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.482902 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5hrd\" (UniqueName: \"kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd\") pod \"authorino-75d556bd77-mk8c6\" (UID: \"b807bb3c-d204-4ca2-8eb3-376fc6a83344\") " pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:31:45.491031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.491005 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5hrd\" (UniqueName: \"kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd\") pod \"authorino-75d556bd77-mk8c6\" (UID: \"b807bb3c-d204-4ca2-8eb3-376fc6a83344\") " pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:31:45.583549 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.583466 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn2m6\" (UniqueName: \"kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6\") pod \"authorino-58cd955675-tngml\" (UID: \"8239aae2-b569-4619-9cb8-4478eef7fa7b\") " pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:45.591628 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.591595 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn2m6\" (UniqueName: \"kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6\") pod \"authorino-58cd955675-tngml\" (UID: \"8239aae2-b569-4619-9cb8-4478eef7fa7b\") " pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:45.639820 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.639789 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sfc8j\"" Jul 06 14:31:45.648832 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.648801 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:45.650019 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.650001 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:31:45.812490 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.812464 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:45.815082 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:31:45.815050 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8239aae2_b569_4619_9cb8_4478eef7fa7b.slice/crio-c6ee799986c3fd57912f7d15ee05e98e9cec9fa3b79d850195a9047b9e77dba1 WatchSource:0}: Error finding container c6ee799986c3fd57912f7d15ee05e98e9cec9fa3b79d850195a9047b9e77dba1: Status 404 returned error can't find the container with id c6ee799986c3fd57912f7d15ee05e98e9cec9fa3b79d850195a9047b9e77dba1 Jul 06 14:31:45.835962 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:45.835940 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:31:45.838344 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:31:45.838301 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb807bb3c_d204_4ca2_8eb3_376fc6a83344.slice/crio-e976a2ac47e030bac003ab145b4b06c274b2de6e942146db8ba6b7d9660f7542 WatchSource:0}: Error finding container e976a2ac47e030bac003ab145b4b06c274b2de6e942146db8ba6b7d9660f7542: Status 404 returned error can't find the container with id e976a2ac47e030bac003ab145b4b06c274b2de6e942146db8ba6b7d9660f7542 Jul 06 14:31:46.248727 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:46.248630 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tngml" event={"ID":"8239aae2-b569-4619-9cb8-4478eef7fa7b","Type":"ContainerStarted","Data":"c6ee799986c3fd57912f7d15ee05e98e9cec9fa3b79d850195a9047b9e77dba1"} Jul 06 14:31:46.249902 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:46.249869 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mk8c6" event={"ID":"b807bb3c-d204-4ca2-8eb3-376fc6a83344","Type":"ContainerStarted","Data":"e976a2ac47e030bac003ab145b4b06c274b2de6e942146db8ba6b7d9660f7542"} Jul 06 14:31:51.283784 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:51.283739 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tngml" event={"ID":"8239aae2-b569-4619-9cb8-4478eef7fa7b","Type":"ContainerStarted","Data":"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a"} Jul 06 14:31:51.285122 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:51.285096 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mk8c6" event={"ID":"b807bb3c-d204-4ca2-8eb3-376fc6a83344","Type":"ContainerStarted","Data":"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc"} Jul 06 14:31:51.302270 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:51.302223 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tngml" podStartSLOduration=2.425511774 podStartE2EDuration="7.302207478s" totalImagesPullingTime="4.876695704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:51.30163253 +0000 UTC m=+546.072654858" watchObservedRunningTime="2026-07-06 14:31:51.302207478 +0000 UTC m=+546.073229809" Jul 06 14:31:51.329126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:51.329060 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mk8c6" podStartSLOduration=2.463466649 podStartE2EDuration="7.329035699s" totalImagesPullingTime="4.86556905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:31:51.326439735 +0000 UTC m=+546.097462067" watchObservedRunningTime="2026-07-06 14:31:51.329035699 +0000 UTC m=+546.100058031" Jul 06 14:31:51.372268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:51.372237 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:53.294786 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:53.294743 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tngml" podUID="8239aae2-b569-4619-9cb8-4478eef7fa7b" containerName="authorino" containerID="cri-o://913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a" gracePeriod=30 Jul 06 14:31:53.548973 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:53.548908 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:53.654115 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:53.654078 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn2m6\" (UniqueName: \"kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6\") pod \"8239aae2-b569-4619-9cb8-4478eef7fa7b\" (UID: \"8239aae2-b569-4619-9cb8-4478eef7fa7b\") " Jul 06 14:31:53.656178 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:53.656148 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6" (OuterVolumeSpecName: "kube-api-access-dn2m6") pod "8239aae2-b569-4619-9cb8-4478eef7fa7b" (UID: "8239aae2-b569-4619-9cb8-4478eef7fa7b"). InnerVolumeSpecName "kube-api-access-dn2m6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:31:53.754691 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:53.754657 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn2m6\" (UniqueName: \"kubernetes.io/projected/8239aae2-b569-4619-9cb8-4478eef7fa7b-kube-api-access-dn2m6\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:31:54.300716 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.300684 2552 generic.go:356] "Generic (PLEG): container finished" podID="8239aae2-b569-4619-9cb8-4478eef7fa7b" containerID="913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a" exitCode=0 Jul 06 14:31:54.301204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.300736 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tngml" Jul 06 14:31:54.301204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.300775 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tngml" event={"ID":"8239aae2-b569-4619-9cb8-4478eef7fa7b","Type":"ContainerDied","Data":"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a"} Jul 06 14:31:54.301204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.300828 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tngml" event={"ID":"8239aae2-b569-4619-9cb8-4478eef7fa7b","Type":"ContainerDied","Data":"c6ee799986c3fd57912f7d15ee05e98e9cec9fa3b79d850195a9047b9e77dba1"} Jul 06 14:31:54.301204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.300852 2552 scope.go:117] "RemoveContainer" containerID="913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a" Jul 06 14:31:54.309571 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.309552 2552 scope.go:117] "RemoveContainer" containerID="913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a" Jul 06 14:31:54.309835 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:31:54.309815 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a\": container with ID starting with 913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a not found: ID does not exist" containerID="913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a" Jul 06 14:31:54.309903 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.309844 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a"} err="failed to get container status \"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a\": rpc error: code = NotFound desc = could not find container \"913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a\": container with ID starting with 913fb08955f68b6586f38e8458280577cb47868938c074ef9ef48fbddc43ee0a not found: ID does not exist" Jul 06 14:31:54.315805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.315780 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:54.319720 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:54.319696 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tngml"] Jul 06 14:31:55.753956 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:31:55.753918 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8239aae2-b569-4619-9cb8-4478eef7fa7b" path="/var/lib/kubelet/pods/8239aae2-b569-4619-9cb8-4478eef7fa7b/volumes" Jul 06 14:32:17.454885 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.454852 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:17.455344 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.455173 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8239aae2-b569-4619-9cb8-4478eef7fa7b" containerName="authorino" Jul 06 14:32:17.455344 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.455185 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8239aae2-b569-4619-9cb8-4478eef7fa7b" containerName="authorino" Jul 06 14:32:17.455344 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.455242 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8239aae2-b569-4619-9cb8-4478eef7fa7b" containerName="authorino" Jul 06 14:32:17.456980 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.456963 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:17.464529 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.464504 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:17.554742 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.554705 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72fgn\" (UniqueName: \"kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn\") pod \"authorino-5d4b67c446-dzsh6\" (UID: \"994f6400-0696-4f2a-8298-d6cbeeefc1ad\") " pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:17.655316 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.655273 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72fgn\" (UniqueName: \"kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn\") pod \"authorino-5d4b67c446-dzsh6\" (UID: \"994f6400-0696-4f2a-8298-d6cbeeefc1ad\") " pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:17.665382 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.665352 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72fgn\" (UniqueName: \"kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn\") pod \"authorino-5d4b67c446-dzsh6\" (UID: \"994f6400-0696-4f2a-8298-d6cbeeefc1ad\") " pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:17.685814 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.685780 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:17.686041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.686029 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:17.707073 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.706992 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d448fdddd-6228p"] Jul 06 14:32:17.709582 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.709561 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:17.719019 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.718990 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d448fdddd-6228p"] Jul 06 14:32:17.831307 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.831279 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:17.834377 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:32:17.834316 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod994f6400_0696_4f2a_8298_d6cbeeefc1ad.slice/crio-1dad7f3ecaba0781295aa2ee0e54d55f2efa2de81f226fee88309a9fe8ccf43e WatchSource:0}: Error finding container 1dad7f3ecaba0781295aa2ee0e54d55f2efa2de81f226fee88309a9fe8ccf43e: Status 404 returned error can't find the container with id 1dad7f3ecaba0781295aa2ee0e54d55f2efa2de81f226fee88309a9fe8ccf43e Jul 06 14:32:17.835038 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.835012 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d448fdddd-6228p"] Jul 06 14:32:17.835281 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:17.835261 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hx94w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7d448fdddd-6228p" podUID="662f5830-78d2-4a4d-be69-1ccfc5d2fd0f" Jul 06 14:32:17.857576 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.857545 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx94w\" (UniqueName: \"kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w\") pod \"authorino-7d448fdddd-6228p\" (UID: \"662f5830-78d2-4a4d-be69-1ccfc5d2fd0f\") " pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:17.958706 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.958607 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx94w\" (UniqueName: \"kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w\") pod \"authorino-7d448fdddd-6228p\" (UID: \"662f5830-78d2-4a4d-be69-1ccfc5d2fd0f\") " pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:17.966644 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:17.966621 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx94w\" (UniqueName: \"kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w\") pod \"authorino-7d448fdddd-6228p\" (UID: \"662f5830-78d2-4a4d-be69-1ccfc5d2fd0f\") " pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:18.423445 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.423407 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" event={"ID":"994f6400-0696-4f2a-8298-d6cbeeefc1ad","Type":"ContainerStarted","Data":"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03"} Jul 06 14:32:18.423445 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.423438 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" podUID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" containerName="authorino" containerID="cri-o://774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03" gracePeriod=30 Jul 06 14:32:18.423931 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.423457 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:18.423931 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.423458 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" event={"ID":"994f6400-0696-4f2a-8298-d6cbeeefc1ad","Type":"ContainerStarted","Data":"1dad7f3ecaba0781295aa2ee0e54d55f2efa2de81f226fee88309a9fe8ccf43e"} Jul 06 14:32:18.430581 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.430558 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:18.442017 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.441971 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" podStartSLOduration=1.02418705 podStartE2EDuration="1.441954711s" totalImagesPullingTime="417.767661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:18.441107944 +0000 UTC m=+573.212130302" watchObservedRunningTime="2026-07-06 14:32:18.441954711 +0000 UTC m=+573.212977041" Jul 06 14:32:18.563255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.563193 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx94w\" (UniqueName: \"kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w\") pod \"662f5830-78d2-4a4d-be69-1ccfc5d2fd0f\" (UID: \"662f5830-78d2-4a4d-be69-1ccfc5d2fd0f\") " Jul 06 14:32:18.565348 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.565305 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w" (OuterVolumeSpecName: "kube-api-access-hx94w") pod "662f5830-78d2-4a4d-be69-1ccfc5d2fd0f" (UID: "662f5830-78d2-4a4d-be69-1ccfc5d2fd0f"). InnerVolumeSpecName "kube-api-access-hx94w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:18.661674 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.661650 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:18.663993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.663975 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx94w\" (UniqueName: \"kubernetes.io/projected/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f-kube-api-access-hx94w\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:32:18.764222 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.764193 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72fgn\" (UniqueName: \"kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn\") pod \"994f6400-0696-4f2a-8298-d6cbeeefc1ad\" (UID: \"994f6400-0696-4f2a-8298-d6cbeeefc1ad\") " Jul 06 14:32:18.766236 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.766200 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn" (OuterVolumeSpecName: "kube-api-access-72fgn") pod "994f6400-0696-4f2a-8298-d6cbeeefc1ad" (UID: "994f6400-0696-4f2a-8298-d6cbeeefc1ad"). InnerVolumeSpecName "kube-api-access-72fgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:18.865546 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:18.865461 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72fgn\" (UniqueName: \"kubernetes.io/projected/994f6400-0696-4f2a-8298-d6cbeeefc1ad-kube-api-access-72fgn\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:32:19.429380 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429342 2552 generic.go:356] "Generic (PLEG): container finished" podID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" containerID="774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03" exitCode=0 Jul 06 14:32:19.429380 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429383 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" Jul 06 14:32:19.429618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429428 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" event={"ID":"994f6400-0696-4f2a-8298-d6cbeeefc1ad","Type":"ContainerDied","Data":"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03"} Jul 06 14:32:19.429618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429469 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dzsh6" event={"ID":"994f6400-0696-4f2a-8298-d6cbeeefc1ad","Type":"ContainerDied","Data":"1dad7f3ecaba0781295aa2ee0e54d55f2efa2de81f226fee88309a9fe8ccf43e"} Jul 06 14:32:19.429618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429485 2552 scope.go:117] "RemoveContainer" containerID="774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03" Jul 06 14:32:19.429732 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.429619 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d448fdddd-6228p" Jul 06 14:32:19.439142 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.439117 2552 scope.go:117] "RemoveContainer" containerID="774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03" Jul 06 14:32:19.439456 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:19.439436 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03\": container with ID starting with 774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03 not found: ID does not exist" containerID="774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03" Jul 06 14:32:19.439527 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.439465 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03"} err="failed to get container status \"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03\": rpc error: code = NotFound desc = could not find container \"774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03\": container with ID starting with 774bf5d68adaff4d81e7580c4f3151564911a48e4f3fad8c0cd5e9a22317ca03 not found: ID does not exist" Jul 06 14:32:19.461712 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.461687 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d448fdddd-6228p"] Jul 06 14:32:19.465606 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.465582 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d448fdddd-6228p"] Jul 06 14:32:19.478622 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.478596 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:19.482685 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.482661 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dzsh6"] Jul 06 14:32:19.754196 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.754117 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="662f5830-78d2-4a4d-be69-1ccfc5d2fd0f" path="/var/lib/kubelet/pods/662f5830-78d2-4a4d-be69-1ccfc5d2fd0f/volumes" Jul 06 14:32:19.754578 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:19.754358 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" path="/var/lib/kubelet/pods/994f6400-0696-4f2a-8298-d6cbeeefc1ad/volumes" Jul 06 14:32:21.971800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:21.971769 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:32:21.972199 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:21.971947 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mk8c6" podUID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" containerName="authorino" containerID="cri-o://6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc" gracePeriod=30 Jul 06 14:32:22.210863 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.210838 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:32:22.297470 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.297443 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5hrd\" (UniqueName: \"kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd\") pod \"b807bb3c-d204-4ca2-8eb3-376fc6a83344\" (UID: \"b807bb3c-d204-4ca2-8eb3-376fc6a83344\") " Jul 06 14:32:22.299537 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.299514 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd" (OuterVolumeSpecName: "kube-api-access-n5hrd") pod "b807bb3c-d204-4ca2-8eb3-376fc6a83344" (UID: "b807bb3c-d204-4ca2-8eb3-376fc6a83344"). InnerVolumeSpecName "kube-api-access-n5hrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:22.398495 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.398454 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5hrd\" (UniqueName: \"kubernetes.io/projected/b807bb3c-d204-4ca2-8eb3-376fc6a83344-kube-api-access-n5hrd\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:32:22.447569 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.447532 2552 generic.go:356] "Generic (PLEG): container finished" podID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" containerID="6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc" exitCode=0 Jul 06 14:32:22.447733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.447579 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mk8c6" Jul 06 14:32:22.447733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.447625 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mk8c6" event={"ID":"b807bb3c-d204-4ca2-8eb3-376fc6a83344","Type":"ContainerDied","Data":"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc"} Jul 06 14:32:22.447733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.447671 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mk8c6" event={"ID":"b807bb3c-d204-4ca2-8eb3-376fc6a83344","Type":"ContainerDied","Data":"e976a2ac47e030bac003ab145b4b06c274b2de6e942146db8ba6b7d9660f7542"} Jul 06 14:32:22.447733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.447688 2552 scope.go:117] "RemoveContainer" containerID="6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc" Jul 06 14:32:22.456375 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.456301 2552 scope.go:117] "RemoveContainer" containerID="6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc" Jul 06 14:32:22.456624 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:22.456606 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc\": container with ID starting with 6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc not found: ID does not exist" containerID="6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc" Jul 06 14:32:22.456667 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.456633 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc"} err="failed to get container status \"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc\": rpc error: code = NotFound desc = could not find container \"6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc\": container with ID starting with 6748b956d3358ba8b082f39343cb584a7d81c5574d2d5bb37a13fb9d5b0f8ebc not found: ID does not exist" Jul 06 14:32:22.467555 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.467528 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:32:22.471208 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:22.471189 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mk8c6"] Jul 06 14:32:23.759356 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:23.759303 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" path="/var/lib/kubelet/pods/b807bb3c-d204-4ca2-8eb3-376fc6a83344/volumes" Jul 06 14:32:26.173153 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173113 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173436 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" containerName="authorino" Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173448 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" containerName="authorino" Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173471 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" containerName="authorino" Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173476 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" containerName="authorino" Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173525 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b807bb3c-d204-4ca2-8eb3-376fc6a83344" containerName="authorino" Jul 06 14:32:26.173544 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.173533 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="994f6400-0696-4f2a-8298-d6cbeeefc1ad" containerName="authorino" Jul 06 14:32:26.176482 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.176465 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.178618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.178588 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:32:26.178618 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.178599 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ll86l\"" Jul 06 14:32:26.178804 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.178601 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:32:26.186711 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.186682 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:26.333976 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.333942 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.334159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.334009 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8pgr\" (UniqueName: \"kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.340159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.340130 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:32:26.342754 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.342736 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.352716 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.352686 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:32:26.435550 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.435452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.435550 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.435516 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8pgr\" (UniqueName: \"kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.435550 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.435544 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42sl\" (UniqueName: \"kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.435774 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:26.435616 2552 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 14:32:26.435774 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.435647 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.435774 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:26.435682 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert podName:883725ad-9cfb-418c-8b30-d3b772c87c43 nodeName:}" failed. No retries permitted until 2026-07-06 14:32:26.935661895 +0000 UTC m=+581.706684203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert") pod "maas-controller-6d8945595c-dbfpk" (UID: "883725ad-9cfb-418c-8b30-d3b772c87c43") : secret "maas-controller-webhook-cert" not found Jul 06 14:32:26.443891 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.443860 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8pgr\" (UniqueName: \"kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.536879 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.536837 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42sl\" (UniqueName: \"kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.537030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.536899 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.539313 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.539287 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.545666 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.545638 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42sl\" (UniqueName: \"kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl\") pod \"maas-controller-74787c7b7f-xcd8s\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.653720 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.653683 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:26.789176 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.789148 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:32:26.791815 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:32:26.791788 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc79098_d81b_40e6_a04d_e78a3bc128ef.slice/crio-ee4b1572eba640db772c4dec5f605411f7a2451589b3e0b954c58301b7243f61 WatchSource:0}: Error finding container ee4b1572eba640db772c4dec5f605411f7a2451589b3e0b954c58301b7243f61: Status 404 returned error can't find the container with id ee4b1572eba640db772c4dec5f605411f7a2451589b3e0b954c58301b7243f61 Jul 06 14:32:26.940957 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.940921 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:26.943460 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:26.943441 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") pod \"maas-controller-6d8945595c-dbfpk\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:27.087738 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:27.087696 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:27.244373 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:27.244349 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:27.246218 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:32:27.246185 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883725ad_9cfb_418c_8b30_d3b772c87c43.slice/crio-cbf9cd7ef47cae14747d227363dbfbcfff35a5f574a8816782f62a7d147c4f0e WatchSource:0}: Error finding container cbf9cd7ef47cae14747d227363dbfbcfff35a5f574a8816782f62a7d147c4f0e: Status 404 returned error can't find the container with id cbf9cd7ef47cae14747d227363dbfbcfff35a5f574a8816782f62a7d147c4f0e Jul 06 14:32:27.475065 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:27.474968 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" event={"ID":"6dc79098-d81b-40e6-a04d-e78a3bc128ef","Type":"ContainerStarted","Data":"ee4b1572eba640db772c4dec5f605411f7a2451589b3e0b954c58301b7243f61"} Jul 06 14:32:27.476832 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:27.476787 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8945595c-dbfpk" event={"ID":"883725ad-9cfb-418c-8b30-d3b772c87c43","Type":"ContainerStarted","Data":"cbf9cd7ef47cae14747d227363dbfbcfff35a5f574a8816782f62a7d147c4f0e"} Jul 06 14:32:29.488421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.488378 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" event={"ID":"6dc79098-d81b-40e6-a04d-e78a3bc128ef","Type":"ContainerStarted","Data":"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c"} Jul 06 14:32:29.488915 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.488530 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:29.490021 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.489992 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8945595c-dbfpk" event={"ID":"883725ad-9cfb-418c-8b30-d3b772c87c43","Type":"ContainerStarted","Data":"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae"} Jul 06 14:32:29.490142 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.490127 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:29.515255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.515202 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" podStartSLOduration=1.299264647 podStartE2EDuration="3.515184308s" totalImagesPullingTime="2.215919661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:29.511595818 +0000 UTC m=+584.282618148" watchObservedRunningTime="2026-07-06 14:32:29.515184308 +0000 UTC m=+584.286206638" Jul 06 14:32:29.533733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:29.533674 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d8945595c-dbfpk" podStartSLOduration=1.761931697 podStartE2EDuration="3.533653105s" totalImagesPullingTime="1.771721408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:32:29.53174882 +0000 UTC m=+584.302771155" watchObservedRunningTime="2026-07-06 14:32:29.533653105 +0000 UTC m=+584.304675436" Jul 06 14:32:40.501240 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.501207 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:32:40.501712 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.501380 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:40.579979 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.579949 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:40.580146 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.580131 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6d8945595c-dbfpk" podUID="883725ad-9cfb-418c-8b30-d3b772c87c43" containerName="manager" containerID="cri-o://86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae" gracePeriod=30 Jul 06 14:32:40.827297 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.827272 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:40.859148 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.859116 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") pod \"883725ad-9cfb-418c-8b30-d3b772c87c43\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " Jul 06 14:32:40.859345 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.859171 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8pgr\" (UniqueName: \"kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr\") pod \"883725ad-9cfb-418c-8b30-d3b772c87c43\" (UID: \"883725ad-9cfb-418c-8b30-d3b772c87c43\") " Jul 06 14:32:40.861512 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.861482 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "883725ad-9cfb-418c-8b30-d3b772c87c43" (UID: "883725ad-9cfb-418c-8b30-d3b772c87c43"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:32:40.861649 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.861580 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr" (OuterVolumeSpecName: "kube-api-access-p8pgr") pod "883725ad-9cfb-418c-8b30-d3b772c87c43" (UID: "883725ad-9cfb-418c-8b30-d3b772c87c43"). InnerVolumeSpecName "kube-api-access-p8pgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:32:40.960557 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.960519 2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883725ad-9cfb-418c-8b30-d3b772c87c43-webhook-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:32:40.960557 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:40.960550 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8pgr\" (UniqueName: \"kubernetes.io/projected/883725ad-9cfb-418c-8b30-d3b772c87c43-kube-api-access-p8pgr\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:32:41.555180 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.555149 2552 generic.go:356] "Generic (PLEG): container finished" podID="883725ad-9cfb-418c-8b30-d3b772c87c43" containerID="86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae" exitCode=0 Jul 06 14:32:41.555630 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.555196 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8945595c-dbfpk" Jul 06 14:32:41.555630 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.555221 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8945595c-dbfpk" event={"ID":"883725ad-9cfb-418c-8b30-d3b772c87c43","Type":"ContainerDied","Data":"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae"} Jul 06 14:32:41.555630 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.555262 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8945595c-dbfpk" event={"ID":"883725ad-9cfb-418c-8b30-d3b772c87c43","Type":"ContainerDied","Data":"cbf9cd7ef47cae14747d227363dbfbcfff35a5f574a8816782f62a7d147c4f0e"} Jul 06 14:32:41.555630 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.555283 2552 scope.go:117] "RemoveContainer" containerID="86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae" Jul 06 14:32:41.564899 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.564883 2552 scope.go:117] "RemoveContainer" containerID="86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae" Jul 06 14:32:41.565167 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:32:41.565150 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae\": container with ID starting with 86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae not found: ID does not exist" containerID="86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae" Jul 06 14:32:41.565208 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.565176 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae"} err="failed to get container status \"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae\": rpc error: code = NotFound desc = could not find container \"86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae\": container with ID starting with 86545a9fadda28e6d7a7a2069fd95450ebf0c0616c870e64d04c9e9ddac5a5ae not found: ID does not exist" Jul 06 14:32:41.585742 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.585700 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:41.585912 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.585760 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d8945595c-dbfpk"] Jul 06 14:32:41.754519 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:41.754484 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883725ad-9cfb-418c-8b30-d3b772c87c43" path="/var/lib/kubelet/pods/883725ad-9cfb-418c-8b30-d3b772c87c43/volumes" Jul 06 14:32:45.657330 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:45.657285 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:32:45.657994 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:45.657978 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:32:45.662071 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:45.662046 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:32:45.662604 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:32:45.662585 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:33:02.908040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.908001 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6ff654ddb6-74kvm"] Jul 06 14:33:02.912419 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.908330 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883725ad-9cfb-418c-8b30-d3b772c87c43" containerName="manager" Jul 06 14:33:02.912419 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.908342 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="883725ad-9cfb-418c-8b30-d3b772c87c43" containerName="manager" Jul 06 14:33:02.912419 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.908406 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="883725ad-9cfb-418c-8b30-d3b772c87c43" containerName="manager" Jul 06 14:33:02.913625 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.913606 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:02.915810 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.915787 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 14:33:02.916175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.916157 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zd4qd\"" Jul 06 14:33:02.928520 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:02.928493 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ff654ddb6-74kvm"] Jul 06 14:33:03.042020 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.041975 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79276d4e-dd7e-4e87-b7f9-764025f69455-openshift-service-ca\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.042020 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.042022 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79276d4e-dd7e-4e87-b7f9-764025f69455-maas-api-tls\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.042269 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.042123 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2r7z\" (UniqueName: \"kubernetes.io/projected/79276d4e-dd7e-4e87-b7f9-764025f69455-kube-api-access-j2r7z\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.143221 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.143185 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79276d4e-dd7e-4e87-b7f9-764025f69455-openshift-service-ca\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.143221 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.143222 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79276d4e-dd7e-4e87-b7f9-764025f69455-maas-api-tls\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.143445 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.143264 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2r7z\" (UniqueName: \"kubernetes.io/projected/79276d4e-dd7e-4e87-b7f9-764025f69455-kube-api-access-j2r7z\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.143992 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.143968 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79276d4e-dd7e-4e87-b7f9-764025f69455-openshift-service-ca\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.146104 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.146077 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79276d4e-dd7e-4e87-b7f9-764025f69455-maas-api-tls\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.151602 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.151569 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2r7z\" (UniqueName: \"kubernetes.io/projected/79276d4e-dd7e-4e87-b7f9-764025f69455-kube-api-access-j2r7z\") pod \"maas-api-6ff654ddb6-74kvm\" (UID: \"79276d4e-dd7e-4e87-b7f9-764025f69455\") " pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.224137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.224048 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:03.370848 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.370807 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ff654ddb6-74kvm"] Jul 06 14:33:03.373354 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:33:03.373308 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79276d4e_dd7e_4e87_b7f9_764025f69455.slice/crio-ad18d86ce5622433f1f4e42aa8d59636879c2c38d8a834bfd50836928f0ee9bf WatchSource:0}: Error finding container ad18d86ce5622433f1f4e42aa8d59636879c2c38d8a834bfd50836928f0ee9bf: Status 404 returned error can't find the container with id ad18d86ce5622433f1f4e42aa8d59636879c2c38d8a834bfd50836928f0ee9bf Jul 06 14:33:03.374748 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.374730 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:33:03.428418 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.428382 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9fkt2"] Jul 06 14:33:03.431379 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.431359 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.433385 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.433365 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 14:33:03.442017 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.441987 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9fkt2"] Jul 06 14:33:03.547313 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.547273 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2560319-f9b2-49e8-8a42-13c6308d9fd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.547313 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.547315 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtf7x\" (UniqueName: \"kubernetes.io/projected/f2560319-f9b2-49e8-8a42-13c6308d9fd2-kube-api-access-rtf7x\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.648374 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.648317 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2560319-f9b2-49e8-8a42-13c6308d9fd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.648374 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.648379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtf7x\" (UniqueName: \"kubernetes.io/projected/f2560319-f9b2-49e8-8a42-13c6308d9fd2-kube-api-access-rtf7x\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.649124 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.649095 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2560319-f9b2-49e8-8a42-13c6308d9fd2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.658970 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.658941 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtf7x\" (UniqueName: \"kubernetes.io/projected/f2560319-f9b2-49e8-8a42-13c6308d9fd2-kube-api-access-rtf7x\") pod \"payload-pre-processing-7964457696-9fkt2\" (UID: \"f2560319-f9b2-49e8-8a42-13c6308d9fd2\") " pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.676664 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.676630 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ff654ddb6-74kvm" event={"ID":"79276d4e-dd7e-4e87-b7f9-764025f69455","Type":"ContainerStarted","Data":"ad18d86ce5622433f1f4e42aa8d59636879c2c38d8a834bfd50836928f0ee9bf"} Jul 06 14:33:03.743024 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.742981 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:03.883227 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:03.883198 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9fkt2"] Jul 06 14:33:03.885349 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:33:03.885292 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2560319_f9b2_49e8_8a42_13c6308d9fd2.slice/crio-893b581f9bdb9d9e1fce4d14b5f233fd21c96bb8b8c147d3f206708c82be1c03 WatchSource:0}: Error finding container 893b581f9bdb9d9e1fce4d14b5f233fd21c96bb8b8c147d3f206708c82be1c03: Status 404 returned error can't find the container with id 893b581f9bdb9d9e1fce4d14b5f233fd21c96bb8b8c147d3f206708c82be1c03 Jul 06 14:33:04.683269 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:04.683230 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" event={"ID":"f2560319-f9b2-49e8-8a42-13c6308d9fd2","Type":"ContainerStarted","Data":"893b581f9bdb9d9e1fce4d14b5f233fd21c96bb8b8c147d3f206708c82be1c03"} Jul 06 14:33:05.689675 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.689634 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ff654ddb6-74kvm" event={"ID":"79276d4e-dd7e-4e87-b7f9-764025f69455","Type":"ContainerStarted","Data":"16414eec8fff64854b866d00f266782fa01424aff0888cb29c66580983d1e37c"} Jul 06 14:33:05.690139 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.689833 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:05.691169 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.691145 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" event={"ID":"f2560319-f9b2-49e8-8a42-13c6308d9fd2","Type":"ContainerStarted","Data":"948467ad19fb35d1c8f3f795db8fbba97719b134c3d924b5141e76b76190def8"} Jul 06 14:33:05.691270 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.691187 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:05.707646 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.707599 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6ff654ddb6-74kvm" podStartSLOduration=1.608639187 podStartE2EDuration="3.707585062s" totalImagesPullingTime="2.098945875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:05.706491429 +0000 UTC m=+620.477513760" watchObservedRunningTime="2026-07-06 14:33:05.707585062 +0000 UTC m=+620.478607393" Jul 06 14:33:05.723241 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:05.723153 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" podStartSLOduration=1.134410101 podStartE2EDuration="2.723138397s" totalImagesPullingTime="1.588728296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:05.721854575 +0000 UTC m=+620.492876902" watchObservedRunningTime="2026-07-06 14:33:05.723138397 +0000 UTC m=+620.494160727" Jul 06 14:33:11.701726 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:11.701656 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6ff654ddb6-74kvm" Jul 06 14:33:16.698784 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:16.698755 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-9fkt2" Jul 06 14:33:27.350179 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.350099 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r"] Jul 06 14:33:27.353438 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.353418 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.355713 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.355686 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 14:33:27.355713 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.355700 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d45hj\"" Jul 06 14:33:27.355945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.355735 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 14:33:27.355945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.355876 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 14:33:27.366759 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.366736 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r"] Jul 06 14:33:27.443879 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.443836 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.443879 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.443867 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.444075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.443889 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.444075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.444003 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.444075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.444039 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.444168 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.444101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54r56\" (UniqueName: \"kubernetes.io/projected/6b413d22-08df-4f3d-be47-81a5cb8e83ec-kube-api-access-54r56\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545010 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.544980 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545010 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545014 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545040 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545073 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545088 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545122 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54r56\" (UniqueName: \"kubernetes.io/projected/6b413d22-08df-4f3d-be47-81a5cb8e83ec-kube-api-access-54r56\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545537 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545514 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545578 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545536 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.545578 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.545564 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.547476 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.547449 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b413d22-08df-4f3d-be47-81a5cb8e83ec-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.547733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.547715 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b413d22-08df-4f3d-be47-81a5cb8e83ec-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.552563 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.552541 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54r56\" (UniqueName: \"kubernetes.io/projected/6b413d22-08df-4f3d-be47-81a5cb8e83ec-kube-api-access-54r56\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r\" (UID: \"6b413d22-08df-4f3d-be47-81a5cb8e83ec\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.666835 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.666733 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:27.811486 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:27.809924 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r"] Jul 06 14:33:28.804213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:28.804149 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" event={"ID":"6b413d22-08df-4f3d-be47-81a5cb8e83ec","Type":"ContainerStarted","Data":"39ea2dd59079bf3239a0b542e2798737dfcbf824b325bfadc87e15a423fdda65"} Jul 06 14:33:29.810952 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:29.810914 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" event={"ID":"6b413d22-08df-4f3d-be47-81a5cb8e83ec","Type":"ContainerStarted","Data":"ef13f63c29ab622e60e34646e952c6e0ea38118110dad7178167e8c0ff05571d"} Jul 06 14:33:29.835206 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:29.835156 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" podStartSLOduration=1.746460407 podStartE2EDuration="2.835141672s" totalImagesPullingTime="1.088681265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:29.833128714 +0000 UTC m=+644.604151056" watchObservedRunningTime="2026-07-06 14:33:29.835141672 +0000 UTC m=+644.606164010" Jul 06 14:33:37.667449 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:37.667395 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:37.667871 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:37.667511 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:37.680127 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:37.680095 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:37.859041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:37.859014 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r" Jul 06 14:33:44.442614 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.442583 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc"] Jul 06 14:33:44.445945 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.445925 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.447975 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.447954 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 14:33:44.456673 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.456650 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc"] Jul 06 14:33:44.478943 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.478910 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzxr6\" (UniqueName: \"kubernetes.io/projected/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-kube-api-access-bzxr6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.479093 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.478969 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.479093 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.479012 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.479093 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.479040 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.479258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.479111 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.479258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.479147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580085 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580051 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580085 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580090 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580354 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580108 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580354 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580139 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580354 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580168 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580354 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580222 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzxr6\" (UniqueName: \"kubernetes.io/projected/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-kube-api-access-bzxr6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580582 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580560 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580658 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580580 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.580712 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.580652 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.582435 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.582402 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.582674 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.582658 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.588913 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.588891 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzxr6\" (UniqueName: \"kubernetes.io/projected/ff3d62d3-d65f-4ebd-acd2-68eb1625356a-kube-api-access-bzxr6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc\" (UID: \"ff3d62d3-d65f-4ebd-acd2-68eb1625356a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.757502 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.757421 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:44.901209 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:44.901182 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc"] Jul 06 14:33:44.902291 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:33:44.902265 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3d62d3_d65f_4ebd_acd2_68eb1625356a.slice/crio-5ced1692a566b1689146fd64b91f253f0548781396663528ff8725ce646c8c19 WatchSource:0}: Error finding container 5ced1692a566b1689146fd64b91f253f0548781396663528ff8725ce646c8c19: Status 404 returned error can't find the container with id 5ced1692a566b1689146fd64b91f253f0548781396663528ff8725ce646c8c19 Jul 06 14:33:45.888392 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:45.888355 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" event={"ID":"ff3d62d3-d65f-4ebd-acd2-68eb1625356a","Type":"ContainerStarted","Data":"5b09f12d4e7423aec3b753c6358d87f331d176a5f0baf2010993dc3a2c0af31c"} Jul 06 14:33:45.888790 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:45.888409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" event={"ID":"ff3d62d3-d65f-4ebd-acd2-68eb1625356a","Type":"ContainerStarted","Data":"5ced1692a566b1689146fd64b91f253f0548781396663528ff8725ce646c8c19"} Jul 06 14:33:45.907991 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:45.907940 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" podStartSLOduration=1.678974705 podStartE2EDuration="1.907925375s" totalImagesPullingTime="228.95067ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:45.905307878 +0000 UTC m=+660.676330204" watchObservedRunningTime="2026-07-06 14:33:45.907925375 +0000 UTC m=+660.678947705" Jul 06 14:33:54.758462 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:54.758417 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:54.758837 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:54.758493 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:54.770979 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:54.770950 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:33:54.948379 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:33:54.948352 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc" Jul 06 14:35:05.361941 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.361899 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79d9cfb94f-28gth"] Jul 06 14:35:05.365436 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.365412 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.367586 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.367568 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 14:35:05.367918 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.367903 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sfc8j\"" Jul 06 14:35:05.375964 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.375935 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d9cfb94f-28gth"] Jul 06 14:35:05.480137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.480096 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cklxx\" (UniqueName: \"kubernetes.io/projected/81862d64-2332-47e8-9cf1-212936fc1ab2-kube-api-access-cklxx\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.480355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.480152 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/81862d64-2332-47e8-9cf1-212936fc1ab2-tls-cert\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.580657 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.580623 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cklxx\" (UniqueName: \"kubernetes.io/projected/81862d64-2332-47e8-9cf1-212936fc1ab2-kube-api-access-cklxx\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.580657 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.580671 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/81862d64-2332-47e8-9cf1-212936fc1ab2-tls-cert\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.583256 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.583226 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/81862d64-2332-47e8-9cf1-212936fc1ab2-tls-cert\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.588801 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.588776 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cklxx\" (UniqueName: \"kubernetes.io/projected/81862d64-2332-47e8-9cf1-212936fc1ab2-kube-api-access-cklxx\") pod \"authorino-79d9cfb94f-28gth\" (UID: \"81862d64-2332-47e8-9cf1-212936fc1ab2\") " pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.675357 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.675239 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d9cfb94f-28gth" Jul 06 14:35:05.819423 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:05.819397 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d9cfb94f-28gth"] Jul 06 14:35:05.821039 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:35:05.821010 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81862d64_2332_47e8_9cf1_212936fc1ab2.slice/crio-756388be705542f4287333df3871857c742bb48a375c87f0268b1ffb788cbd4c WatchSource:0}: Error finding container 756388be705542f4287333df3871857c742bb48a375c87f0268b1ffb788cbd4c: Status 404 returned error can't find the container with id 756388be705542f4287333df3871857c742bb48a375c87f0268b1ffb788cbd4c Jul 06 14:35:06.296362 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:06.296293 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d9cfb94f-28gth" event={"ID":"81862d64-2332-47e8-9cf1-212936fc1ab2","Type":"ContainerStarted","Data":"64df546c31a82e1934a25e4e7b68579633d4fdef267694b495e918105034deba"} Jul 06 14:35:06.296549 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:06.296376 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d9cfb94f-28gth" event={"ID":"81862d64-2332-47e8-9cf1-212936fc1ab2","Type":"ContainerStarted","Data":"756388be705542f4287333df3871857c742bb48a375c87f0268b1ffb788cbd4c"} Jul 06 14:35:06.328221 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:35:06.328156 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79d9cfb94f-28gth" podStartSLOduration=0.959833477 podStartE2EDuration="1.328134964s" totalImagesPullingTime="368.301487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:35:06.325981669 +0000 UTC m=+741.097004013" watchObservedRunningTime="2026-07-06 14:35:06.328134964 +0000 UTC m=+741.099157295" Jul 06 14:36:17.958307 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:17.958220 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:36:17.958814 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:17.958446 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" podUID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" containerName="manager" containerID="cri-o://e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c" gracePeriod=30 Jul 06 14:36:18.210121 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.210058 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:36:18.395046 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.395010 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k42sl\" (UniqueName: \"kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl\") pod \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " Jul 06 14:36:18.395223 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.395081 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert\") pod \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\" (UID: \"6dc79098-d81b-40e6-a04d-e78a3bc128ef\") " Jul 06 14:36:18.397155 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.397132 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl" (OuterVolumeSpecName: "kube-api-access-k42sl") pod "6dc79098-d81b-40e6-a04d-e78a3bc128ef" (UID: "6dc79098-d81b-40e6-a04d-e78a3bc128ef"). InnerVolumeSpecName "kube-api-access-k42sl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:36:18.397155 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.397140 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6dc79098-d81b-40e6-a04d-e78a3bc128ef" (UID: "6dc79098-d81b-40e6-a04d-e78a3bc128ef"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:36:18.496555 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.496466 2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6dc79098-d81b-40e6-a04d-e78a3bc128ef-webhook-cert\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:36:18.496555 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.496503 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k42sl\" (UniqueName: \"kubernetes.io/projected/6dc79098-d81b-40e6-a04d-e78a3bc128ef-kube-api-access-k42sl\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:36:18.681485 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.681450 2552 generic.go:356] "Generic (PLEG): container finished" podID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" containerID="e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c" exitCode=0 Jul 06 14:36:18.681485 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.681487 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" event={"ID":"6dc79098-d81b-40e6-a04d-e78a3bc128ef","Type":"ContainerDied","Data":"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c"} Jul 06 14:36:18.681690 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.681494 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" Jul 06 14:36:18.681690 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.681528 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-xcd8s" event={"ID":"6dc79098-d81b-40e6-a04d-e78a3bc128ef","Type":"ContainerDied","Data":"ee4b1572eba640db772c4dec5f605411f7a2451589b3e0b954c58301b7243f61"} Jul 06 14:36:18.681690 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.681551 2552 scope.go:117] "RemoveContainer" containerID="e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c" Jul 06 14:36:18.690796 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.690780 2552 scope.go:117] "RemoveContainer" containerID="e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c" Jul 06 14:36:18.691055 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:36:18.691035 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c\": container with ID starting with e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c not found: ID does not exist" containerID="e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c" Jul 06 14:36:18.691128 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.691066 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c"} err="failed to get container status \"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c\": rpc error: code = NotFound desc = could not find container \"e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c\": container with ID starting with e285c74b1e8a4df317eaba4bdecd02f9393432bd79b198dd536caae2bb0e804c not found: ID does not exist" Jul 06 14:36:18.707233 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.707210 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:36:18.711726 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:18.711701 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-xcd8s"] Jul 06 14:36:19.573662 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.573630 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74787c7b7f-rd2ht"] Jul 06 14:36:19.574028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.573995 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" containerName="manager" Jul 06 14:36:19.574028 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.574005 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" containerName="manager" Jul 06 14:36:19.574098 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.574062 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" containerName="manager" Jul 06 14:36:19.578148 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.578131 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.583633 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.583609 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ll86l\"" Jul 06 14:36:19.583768 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.583612 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:36:19.583768 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.583614 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:36:19.603145 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.603119 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-rd2ht"] Jul 06 14:36:19.707095 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.707059 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g99s\" (UniqueName: \"kubernetes.io/projected/5c7036fb-2724-407b-80cb-69816c1a17cc-kube-api-access-4g99s\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.707278 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.707104 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c7036fb-2724-407b-80cb-69816c1a17cc-webhook-cert\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.753724 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.753683 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dc79098-d81b-40e6-a04d-e78a3bc128ef" path="/var/lib/kubelet/pods/6dc79098-d81b-40e6-a04d-e78a3bc128ef/volumes" Jul 06 14:36:19.807613 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.807582 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g99s\" (UniqueName: \"kubernetes.io/projected/5c7036fb-2724-407b-80cb-69816c1a17cc-kube-api-access-4g99s\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.807805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.807628 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c7036fb-2724-407b-80cb-69816c1a17cc-webhook-cert\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.810045 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.810021 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c7036fb-2724-407b-80cb-69816c1a17cc-webhook-cert\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.819845 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.819820 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g99s\" (UniqueName: \"kubernetes.io/projected/5c7036fb-2724-407b-80cb-69816c1a17cc-kube-api-access-4g99s\") pod \"maas-controller-74787c7b7f-rd2ht\" (UID: \"5c7036fb-2724-407b-80cb-69816c1a17cc\") " pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:19.888676 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:19.888593 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:20.043995 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:20.043964 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74787c7b7f-rd2ht"] Jul 06 14:36:20.044740 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:36:20.044711 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7036fb_2724_407b_80cb_69816c1a17cc.slice/crio-c6b52acdca71c30cf6cd950143a28dea11bf55b3a6a3c5ed899cde03d4f52440 WatchSource:0}: Error finding container c6b52acdca71c30cf6cd950143a28dea11bf55b3a6a3c5ed899cde03d4f52440: Status 404 returned error can't find the container with id c6b52acdca71c30cf6cd950143a28dea11bf55b3a6a3c5ed899cde03d4f52440 Jul 06 14:36:20.694164 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:20.694126 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" event={"ID":"5c7036fb-2724-407b-80cb-69816c1a17cc","Type":"ContainerStarted","Data":"41fea30570c8eb0d2c0044be69819b97feeef55d11abbaf0e1b23579a4189cd5"} Jul 06 14:36:20.694646 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:20.694184 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" event={"ID":"5c7036fb-2724-407b-80cb-69816c1a17cc","Type":"ContainerStarted","Data":"c6b52acdca71c30cf6cd950143a28dea11bf55b3a6a3c5ed899cde03d4f52440"} Jul 06 14:36:20.694646 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:20.694210 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:36:20.727391 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:20.727341 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" podStartSLOduration=1.247331153 podStartE2EDuration="1.727312388s" totalImagesPullingTime="479.981235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:36:20.72604349 +0000 UTC m=+815.497065822" watchObservedRunningTime="2026-07-06 14:36:20.727312388 +0000 UTC m=+815.498334717" Jul 06 14:36:31.705724 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:36:31.705692 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74787c7b7f-rd2ht" Jul 06 14:37:45.689897 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:37:45.689822 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:37:45.692841 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:37:45.692810 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:37:45.697482 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:37:45.697454 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:37:45.700080 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:37:45.700058 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:42:45.727182 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:42:45.727146 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:42:45.731606 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:42:45.731579 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:42:45.733519 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:42:45.733494 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:42:45.737723 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:42:45.737703 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:47:45.764837 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:47:45.764810 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:47:45.769178 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:47:45.769153 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:47:45.773807 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:47:45.773787 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:47:45.778088 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:47:45.778066 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:48:48.516733 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.516698 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:48.520913 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.520892 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.523310 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.523286 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-7ebf541d-serving-cert\"" Jul 06 14:48:48.530927 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.530892 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:48.571100 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.571061 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kn7l\" (UniqueName: \"kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.571294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.571151 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.571294 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.571189 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.672597 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.672559 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kn7l\" (UniqueName: \"kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.672777 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.672612 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.672777 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.672635 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.673410 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.673375 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.675064 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.675043 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.680355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.680335 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kn7l\" (UniqueName: \"kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l\") pod \"maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.832775 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.832730 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:48.979695 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.975872 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:48.981652 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:48:48.981622 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a34984e_2cea_4113_a40c_d8b08f92d399.slice/crio-3c4ac3de6e77cd56c4c81601a8f64ee2a2cb82f6d8ff03c3f711052703243de4 WatchSource:0}: Error finding container 3c4ac3de6e77cd56c4c81601a8f64ee2a2cb82f6d8ff03c3f711052703243de4: Status 404 returned error can't find the container with id 3c4ac3de6e77cd56c4c81601a8f64ee2a2cb82f6d8ff03c3f711052703243de4 Jul 06 14:48:48.983389 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:48.983371 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:48:49.572659 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:49.572618 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" event={"ID":"8a34984e-2cea-4113-a40c-d8b08f92d399","Type":"ContainerStarted","Data":"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6"} Jul 06 14:48:49.573072 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:49.572687 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" event={"ID":"8a34984e-2cea-4113-a40c-d8b08f92d399","Type":"ContainerStarted","Data":"3c4ac3de6e77cd56c4c81601a8f64ee2a2cb82f6d8ff03c3f711052703243de4"} Jul 06 14:48:49.573072 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:49.572745 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:49.596830 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:49.596774 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" podStartSLOduration=1.125842017 podStartE2EDuration="1.596757573s" totalImagesPullingTime="470.915556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:48:49.594395635 +0000 UTC m=+1564.365417968" watchObservedRunningTime="2026-07-06 14:48:49.596757573 +0000 UTC m=+1564.367779920" Jul 06 14:48:54.029602 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.029566 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:54.030031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.029807 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" podUID="8a34984e-2cea-4113-a40c-d8b08f92d399" containerName="maas-api" containerID="cri-o://446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6" gracePeriod=30 Jul 06 14:48:54.035439 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.035413 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:54.280750 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.280692 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:54.326040 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.326008 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kn7l\" (UniqueName: \"kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l\") pod \"8a34984e-2cea-4113-a40c-d8b08f92d399\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " Jul 06 14:48:54.326227 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.326127 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls\") pod \"8a34984e-2cea-4113-a40c-d8b08f92d399\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " Jul 06 14:48:54.326227 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.326160 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca\") pod \"8a34984e-2cea-4113-a40c-d8b08f92d399\" (UID: \"8a34984e-2cea-4113-a40c-d8b08f92d399\") " Jul 06 14:48:54.326568 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.326546 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a34984e-2cea-4113-a40c-d8b08f92d399" (UID: "8a34984e-2cea-4113-a40c-d8b08f92d399"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:48:54.328115 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.328083 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l" (OuterVolumeSpecName: "kube-api-access-7kn7l") pod "8a34984e-2cea-4113-a40c-d8b08f92d399" (UID: "8a34984e-2cea-4113-a40c-d8b08f92d399"). InnerVolumeSpecName "kube-api-access-7kn7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:48:54.328215 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.328197 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a34984e-2cea-4113-a40c-d8b08f92d399" (UID: "8a34984e-2cea-4113-a40c-d8b08f92d399"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:48:54.427624 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.427589 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a34984e-2cea-4113-a40c-d8b08f92d399-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:48:54.427624 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.427617 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a34984e-2cea-4113-a40c-d8b08f92d399-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:48:54.427624 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.427626 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kn7l\" (UniqueName: \"kubernetes.io/projected/8a34984e-2cea-4113-a40c-d8b08f92d399-kube-api-access-7kn7l\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:48:54.599652 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.599616 2552 generic.go:356] "Generic (PLEG): container finished" podID="8a34984e-2cea-4113-a40c-d8b08f92d399" containerID="446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6" exitCode=0 Jul 06 14:48:54.599834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.599660 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" Jul 06 14:48:54.599834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.599697 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" event={"ID":"8a34984e-2cea-4113-a40c-d8b08f92d399","Type":"ContainerDied","Data":"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6"} Jul 06 14:48:54.599834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.599750 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq" event={"ID":"8a34984e-2cea-4113-a40c-d8b08f92d399","Type":"ContainerDied","Data":"3c4ac3de6e77cd56c4c81601a8f64ee2a2cb82f6d8ff03c3f711052703243de4"} Jul 06 14:48:54.599834 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.599774 2552 scope.go:117] "RemoveContainer" containerID="446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6" Jul 06 14:48:54.609687 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.609669 2552 scope.go:117] "RemoveContainer" containerID="446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6" Jul 06 14:48:54.609923 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:48:54.609902 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6\": container with ID starting with 446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6 not found: ID does not exist" containerID="446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6" Jul 06 14:48:54.609986 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.609935 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6"} err="failed to get container status \"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6\": rpc error: code = NotFound desc = could not find container \"446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6\": container with ID starting with 446c1ece16950672f067a633acc0cbeab1a18370ef8fb2a6bb624883e419a8c6 not found: ID does not exist" Jul 06 14:48:54.624403 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.624378 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:54.633909 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:54.633883 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-7ebf541d-56fbd5bc59-6nmjq"] Jul 06 14:48:55.754583 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:48:55.754547 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a34984e-2cea-4113-a40c-d8b08f92d399" path="/var/lib/kubelet/pods/8a34984e-2cea-4113-a40c-d8b08f92d399/volumes" Jul 06 14:49:01.432983 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.432949 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:01.433520 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.433500 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a34984e-2cea-4113-a40c-d8b08f92d399" containerName="maas-api" Jul 06 14:49:01.433603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.433524 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a34984e-2cea-4113-a40c-d8b08f92d399" containerName="maas-api" Jul 06 14:49:01.433655 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.433614 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a34984e-2cea-4113-a40c-d8b08f92d399" containerName="maas-api" Jul 06 14:49:01.437804 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.437782 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.439864 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.439831 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-1d78acdd-serving-cert\"" Jul 06 14:49:01.453682 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.453643 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:01.592703 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.592668 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.592872 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.592715 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.592872 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.592784 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkzpv\" (UniqueName: \"kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.693766 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.693668 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.693766 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.693759 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkzpv\" (UniqueName: \"kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.694011 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.693808 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.694011 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:01.693933 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-1d78acdd-serving-cert: secret "maas-api-e2e-ait-1d78acdd-serving-cert" not found Jul 06 14:49:01.694011 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:01.694009 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls podName:a048fb0b-1575-499b-85b9-635f0add097b nodeName:}" failed. No retries permitted until 2026-07-06 14:49:02.193986519 +0000 UTC m=+1576.965008844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls") pod "maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" (UID: "a048fb0b-1575-499b-85b9-635f0add097b") : secret "maas-api-e2e-ait-1d78acdd-serving-cert" not found Jul 06 14:49:01.694355 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.694306 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:01.708365 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:01.708315 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkzpv\" (UniqueName: \"kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:02.198724 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:02.198683 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:02.201087 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:02.201065 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") pod \"maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:02.353473 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:02.353437 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:02.494412 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:02.494383 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:02.496522 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:49:02.496493 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda048fb0b_1575_499b_85b9_635f0add097b.slice/crio-d21e06b7eb96a0133589b3808c517dd21b78649abac8660895c3536b10bbc9b3 WatchSource:0}: Error finding container d21e06b7eb96a0133589b3808c517dd21b78649abac8660895c3536b10bbc9b3: Status 404 returned error can't find the container with id d21e06b7eb96a0133589b3808c517dd21b78649abac8660895c3536b10bbc9b3 Jul 06 14:49:02.643809 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:02.643758 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" event={"ID":"a048fb0b-1575-499b-85b9-635f0add097b","Type":"ContainerStarted","Data":"d21e06b7eb96a0133589b3808c517dd21b78649abac8660895c3536b10bbc9b3"} Jul 06 14:49:03.650950 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:03.650856 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" event={"ID":"a048fb0b-1575-499b-85b9-635f0add097b","Type":"ContainerStarted","Data":"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96"} Jul 06 14:49:03.651293 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:03.650995 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:03.668295 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:03.668241 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" podStartSLOduration=2.238399733 podStartE2EDuration="2.668224855s" totalImagesPullingTime="429.825122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:03.667474483 +0000 UTC m=+1578.438496841" watchObservedRunningTime="2026-07-06 14:49:03.668224855 +0000 UTC m=+1578.439247182" Jul 06 14:49:06.951111 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:06.951066 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:06.951577 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:06.951313 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" podUID="a048fb0b-1575-499b-85b9-635f0add097b" containerName="maas-api" containerID="cri-o://d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96" gracePeriod=30 Jul 06 14:49:07.214711 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.214633 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:07.345027 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.344996 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkzpv\" (UniqueName: \"kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv\") pod \"a048fb0b-1575-499b-85b9-635f0add097b\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " Jul 06 14:49:07.345027 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.345036 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca\") pod \"a048fb0b-1575-499b-85b9-635f0add097b\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " Jul 06 14:49:07.345245 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.345088 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") pod \"a048fb0b-1575-499b-85b9-635f0add097b\" (UID: \"a048fb0b-1575-499b-85b9-635f0add097b\") " Jul 06 14:49:07.345475 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.345447 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a048fb0b-1575-499b-85b9-635f0add097b" (UID: "a048fb0b-1575-499b-85b9-635f0add097b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:07.347175 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.347136 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a048fb0b-1575-499b-85b9-635f0add097b" (UID: "a048fb0b-1575-499b-85b9-635f0add097b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:49:07.347283 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.347215 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv" (OuterVolumeSpecName: "kube-api-access-zkzpv") pod "a048fb0b-1575-499b-85b9-635f0add097b" (UID: "a048fb0b-1575-499b-85b9-635f0add097b"). InnerVolumeSpecName "kube-api-access-zkzpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:07.446586 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.446551 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a048fb0b-1575-499b-85b9-635f0add097b-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:07.446586 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.446580 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkzpv\" (UniqueName: \"kubernetes.io/projected/a048fb0b-1575-499b-85b9-635f0add097b-kube-api-access-zkzpv\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:07.446586 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.446590 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a048fb0b-1575-499b-85b9-635f0add097b-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:07.672677 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.672643 2552 generic.go:356] "Generic (PLEG): container finished" podID="a048fb0b-1575-499b-85b9-635f0add097b" containerID="d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96" exitCode=0 Jul 06 14:49:07.672833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.672687 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" Jul 06 14:49:07.672833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.672701 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" event={"ID":"a048fb0b-1575-499b-85b9-635f0add097b","Type":"ContainerDied","Data":"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96"} Jul 06 14:49:07.672833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.672742 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj" event={"ID":"a048fb0b-1575-499b-85b9-635f0add097b","Type":"ContainerDied","Data":"d21e06b7eb96a0133589b3808c517dd21b78649abac8660895c3536b10bbc9b3"} Jul 06 14:49:07.672833 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.672762 2552 scope.go:117] "RemoveContainer" containerID="d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96" Jul 06 14:49:07.682602 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.682581 2552 scope.go:117] "RemoveContainer" containerID="d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96" Jul 06 14:49:07.682863 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:07.682845 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96\": container with ID starting with d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96 not found: ID does not exist" containerID="d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96" Jul 06 14:49:07.682925 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.682873 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96"} err="failed to get container status \"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96\": rpc error: code = NotFound desc = could not find container \"d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96\": container with ID starting with d727b236713b06efb0bac456817f4974cfaac8a3fd3e18dc208ebd0fbebc9b96 not found: ID does not exist" Jul 06 14:49:07.694199 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.694175 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:07.698393 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.698372 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-1d78acdd-5bcb8d7689-ghtnj"] Jul 06 14:49:07.755293 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:07.755261 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a048fb0b-1575-499b-85b9-635f0add097b" path="/var/lib/kubelet/pods/a048fb0b-1575-499b-85b9-635f0add097b/volumes" Jul 06 14:49:20.583269 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.583237 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:20.583693 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.583605 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a048fb0b-1575-499b-85b9-635f0add097b" containerName="maas-api" Jul 06 14:49:20.583693 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.583618 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a048fb0b-1575-499b-85b9-635f0add097b" containerName="maas-api" Jul 06 14:49:20.583766 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.583708 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a048fb0b-1575-499b-85b9-635f0add097b" containerName="maas-api" Jul 06 14:49:20.586666 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.586649 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.590127 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.590106 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-b7b274f6-serving-cert\"" Jul 06 14:49:20.603690 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.603658 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:20.662416 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.662377 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dxw\" (UniqueName: \"kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.662416 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.662420 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.662662 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.662515 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.762918 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.762880 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.763111 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.762945 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dxw\" (UniqueName: \"kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.763111 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.762971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.763111 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:20.762987 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-b7b274f6-serving-cert: secret "maas-api-e2e-derive-b7b274f6-serving-cert" not found Jul 06 14:49:20.763111 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:20.763072 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls podName:773fc030-620d-4f3f-add3-3868667bd314 nodeName:}" failed. No retries permitted until 2026-07-06 14:49:21.263054505 +0000 UTC m=+1596.034076817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls") pod "maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" (UID: "773fc030-620d-4f3f-add3-3868667bd314") : secret "maas-api-e2e-derive-b7b274f6-serving-cert" not found Jul 06 14:49:20.763663 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.763646 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:20.771808 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:20.771779 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dxw\" (UniqueName: \"kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:21.267452 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:21.267415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:21.269866 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:21.269842 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") pod \"maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:21.496487 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:21.496454 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:21.635303 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:21.635277 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:21.637258 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:49:21.637226 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773fc030_620d_4f3f_add3_3868667bd314.slice/crio-afce089dc7ad8873960a262b44adbe4a64890c076235573440f742875ba4b640 WatchSource:0}: Error finding container afce089dc7ad8873960a262b44adbe4a64890c076235573440f742875ba4b640: Status 404 returned error can't find the container with id afce089dc7ad8873960a262b44adbe4a64890c076235573440f742875ba4b640 Jul 06 14:49:21.746312 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:21.746269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" event={"ID":"773fc030-620d-4f3f-add3-3868667bd314","Type":"ContainerStarted","Data":"afce089dc7ad8873960a262b44adbe4a64890c076235573440f742875ba4b640"} Jul 06 14:49:22.753116 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:22.753078 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" event={"ID":"773fc030-620d-4f3f-add3-3868667bd314","Type":"ContainerStarted","Data":"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684"} Jul 06 14:49:22.753560 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:22.753180 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:22.771752 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:22.771702 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" podStartSLOduration=2.316823355 podStartE2EDuration="2.771684766s" totalImagesPullingTime="454.861411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:22.768993598 +0000 UTC m=+1597.540015929" watchObservedRunningTime="2026-07-06 14:49:22.771684766 +0000 UTC m=+1597.542707096" Jul 06 14:49:25.084155 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.084114 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:25.084675 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.084383 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" podUID="773fc030-620d-4f3f-add3-3868667bd314" containerName="maas-api" containerID="cri-o://c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684" gracePeriod=30 Jul 06 14:49:25.338993 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.338932 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:25.396476 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.396439 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca\") pod \"773fc030-620d-4f3f-add3-3868667bd314\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " Jul 06 14:49:25.396636 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.396488 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") pod \"773fc030-620d-4f3f-add3-3868667bd314\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " Jul 06 14:49:25.396636 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.396524 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5dxw\" (UniqueName: \"kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw\") pod \"773fc030-620d-4f3f-add3-3868667bd314\" (UID: \"773fc030-620d-4f3f-add3-3868667bd314\") " Jul 06 14:49:25.396835 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.396810 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "773fc030-620d-4f3f-add3-3868667bd314" (UID: "773fc030-620d-4f3f-add3-3868667bd314"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:25.398636 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.398609 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw" (OuterVolumeSpecName: "kube-api-access-n5dxw") pod "773fc030-620d-4f3f-add3-3868667bd314" (UID: "773fc030-620d-4f3f-add3-3868667bd314"). InnerVolumeSpecName "kube-api-access-n5dxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:25.398760 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.398641 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "773fc030-620d-4f3f-add3-3868667bd314" (UID: "773fc030-620d-4f3f-add3-3868667bd314"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:49:25.497204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.497165 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/773fc030-620d-4f3f-add3-3868667bd314-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:25.497204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.497197 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/773fc030-620d-4f3f-add3-3868667bd314-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:25.497204 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.497208 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5dxw\" (UniqueName: \"kubernetes.io/projected/773fc030-620d-4f3f-add3-3868667bd314-kube-api-access-n5dxw\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:49:25.772059 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.772023 2552 generic.go:356] "Generic (PLEG): container finished" podID="773fc030-620d-4f3f-add3-3868667bd314" containerID="c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684" exitCode=0 Jul 06 14:49:25.772203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.772070 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" Jul 06 14:49:25.772203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.772102 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" event={"ID":"773fc030-620d-4f3f-add3-3868667bd314","Type":"ContainerDied","Data":"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684"} Jul 06 14:49:25.772203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.772159 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf" event={"ID":"773fc030-620d-4f3f-add3-3868667bd314","Type":"ContainerDied","Data":"afce089dc7ad8873960a262b44adbe4a64890c076235573440f742875ba4b640"} Jul 06 14:49:25.772203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.772183 2552 scope.go:117] "RemoveContainer" containerID="c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684" Jul 06 14:49:25.780784 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.780753 2552 scope.go:117] "RemoveContainer" containerID="c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684" Jul 06 14:49:25.781075 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:49:25.781054 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684\": container with ID starting with c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684 not found: ID does not exist" containerID="c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684" Jul 06 14:49:25.781137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.781090 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684"} err="failed to get container status \"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684\": rpc error: code = NotFound desc = could not find container \"c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684\": container with ID starting with c20bf3586b2e8c28b1fb6874949a74a05f95e880b79b4244f483beba15d2f684 not found: ID does not exist" Jul 06 14:49:25.789291 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.789265 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:25.796021 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:25.795999 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-b7b274f6-6b55dc9ffd-5mwkf"] Jul 06 14:49:27.754889 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:49:27.754855 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="773fc030-620d-4f3f-add3-3868667bd314" path="/var/lib/kubelet/pods/773fc030-620d-4f3f-add3-3868667bd314/volumes" Jul 06 14:51:16.453285 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.453197 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:16.453856 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.453756 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="773fc030-620d-4f3f-add3-3868667bd314" containerName="maas-api" Jul 06 14:51:16.453856 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.453778 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="773fc030-620d-4f3f-add3-3868667bd314" containerName="maas-api" Jul 06 14:51:16.453983 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.453896 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="773fc030-620d-4f3f-add3-3868667bd314" containerName="maas-api" Jul 06 14:51:16.456864 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.456844 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.462417 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.462392 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-234d4ebd-serving-cert\"" Jul 06 14:51:16.481954 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.481922 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:16.510627 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.510592 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpp64\" (UniqueName: \"kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.510800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.510662 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.510800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.510696 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.611811 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.611772 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.612004 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.611831 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.612004 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.611871 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpp64\" (UniqueName: \"kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.612137 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:51:16.612021 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-234d4ebd-serving-cert: secret "maas-api-e2e-mt-234d4ebd-serving-cert" not found Jul 06 14:51:16.612137 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:51:16.612116 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls podName:9bae11d1-40ce-4c89-9feb-71a62b5765b4 nodeName:}" failed. No retries permitted until 2026-07-06 14:51:17.112092463 +0000 UTC m=+1711.883114798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls") pod "maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" (UID: "9bae11d1-40ce-4c89-9feb-71a62b5765b4") : secret "maas-api-e2e-mt-234d4ebd-serving-cert" not found Jul 06 14:51:16.612550 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.612528 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:16.629735 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:16.629703 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpp64\" (UniqueName: \"kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:17.116839 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:17.116800 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:17.119226 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:17.119205 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") pod \"maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:17.367405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:17.367293 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:17.527861 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:17.527833 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:17.530154 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:51:17.530122 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bae11d1_40ce_4c89_9feb_71a62b5765b4.slice/crio-3551ac0ffdb9bd2e8e1707e90e1694640b3f55f632d9d745e751172695d16248 WatchSource:0}: Error finding container 3551ac0ffdb9bd2e8e1707e90e1694640b3f55f632d9d745e751172695d16248: Status 404 returned error can't find the container with id 3551ac0ffdb9bd2e8e1707e90e1694640b3f55f632d9d745e751172695d16248 Jul 06 14:51:18.372337 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:18.372282 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" event={"ID":"9bae11d1-40ce-4c89-9feb-71a62b5765b4","Type":"ContainerStarted","Data":"b6a1956b9068ec041505af211391196bdfe1cfdfec99271dafd8888286969eb2"} Jul 06 14:51:18.372543 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:18.372517 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:18.372611 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:18.372553 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" event={"ID":"9bae11d1-40ce-4c89-9feb-71a62b5765b4","Type":"ContainerStarted","Data":"3551ac0ffdb9bd2e8e1707e90e1694640b3f55f632d9d745e751172695d16248"} Jul 06 14:51:18.396186 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:18.396129 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" podStartSLOduration=1.939592837 podStartE2EDuration="2.396112813s" totalImagesPullingTime="456.519976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:51:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:51:18.392877445 +0000 UTC m=+1713.163899777" watchObservedRunningTime="2026-07-06 14:51:18.396112813 +0000 UTC m=+1713.167135143" Jul 06 14:51:24.384410 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:24.384378 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:29.303785 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.303745 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:29.304189 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.303944 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerName="maas-api" containerID="cri-o://b6a1956b9068ec041505af211391196bdfe1cfdfec99271dafd8888286969eb2" gracePeriod=30 Jul 06 14:51:29.379005 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.378969 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.51:8443/health\": dial tcp 10.132.0.51:8443: connect: connection refused" Jul 06 14:51:29.441435 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.441397 2552 generic.go:356] "Generic (PLEG): container finished" podID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerID="b6a1956b9068ec041505af211391196bdfe1cfdfec99271dafd8888286969eb2" exitCode=0 Jul 06 14:51:29.441617 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.441488 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" event={"ID":"9bae11d1-40ce-4c89-9feb-71a62b5765b4","Type":"ContainerDied","Data":"b6a1956b9068ec041505af211391196bdfe1cfdfec99271dafd8888286969eb2"} Jul 06 14:51:29.565579 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.565507 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:29.725268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.725232 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") pod \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " Jul 06 14:51:29.725268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.725275 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca\") pod \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " Jul 06 14:51:29.725552 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.725481 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpp64\" (UniqueName: \"kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64\") pod \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\" (UID: \"9bae11d1-40ce-4c89-9feb-71a62b5765b4\") " Jul 06 14:51:29.725670 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.725649 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9bae11d1-40ce-4c89-9feb-71a62b5765b4" (UID: "9bae11d1-40ce-4c89-9feb-71a62b5765b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:51:29.725735 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.725717 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bae11d1-40ce-4c89-9feb-71a62b5765b4-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:51:29.727478 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.727457 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64" (OuterVolumeSpecName: "kube-api-access-cpp64") pod "9bae11d1-40ce-4c89-9feb-71a62b5765b4" (UID: "9bae11d1-40ce-4c89-9feb-71a62b5765b4"). InnerVolumeSpecName "kube-api-access-cpp64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:51:29.727571 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.727552 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9bae11d1-40ce-4c89-9feb-71a62b5765b4" (UID: "9bae11d1-40ce-4c89-9feb-71a62b5765b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:51:29.826800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.826713 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpp64\" (UniqueName: \"kubernetes.io/projected/9bae11d1-40ce-4c89-9feb-71a62b5765b4-kube-api-access-cpp64\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:51:29.826800 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:29.826745 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bae11d1-40ce-4c89-9feb-71a62b5765b4-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:51:30.448043 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:30.448012 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" Jul 06 14:51:30.448043 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:30.448019 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v" event={"ID":"9bae11d1-40ce-4c89-9feb-71a62b5765b4","Type":"ContainerDied","Data":"3551ac0ffdb9bd2e8e1707e90e1694640b3f55f632d9d745e751172695d16248"} Jul 06 14:51:30.448580 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:30.448084 2552 scope.go:117] "RemoveContainer" containerID="b6a1956b9068ec041505af211391196bdfe1cfdfec99271dafd8888286969eb2" Jul 06 14:51:30.466292 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:30.466268 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:30.474487 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:30.474466 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-234d4ebd-5c6cb56b6-8ps5v"] Jul 06 14:51:31.754389 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:51:31.754352 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" path="/var/lib/kubelet/pods/9bae11d1-40ce-4c89-9feb-71a62b5765b4/volumes" Jul 06 14:52:02.902556 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.902520 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:02.903154 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.902927 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerName="maas-api" Jul 06 14:52:02.903154 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.902945 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerName="maas-api" Jul 06 14:52:02.903154 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.903062 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bae11d1-40ce-4c89-9feb-71a62b5765b4" containerName="maas-api" Jul 06 14:52:02.906100 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.906075 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:02.909107 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.909088 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4937319b-serving-cert\"" Jul 06 14:52:02.920562 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:02.920534 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:03.001286 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.001242 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpkqq\" (UniqueName: \"kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.001479 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.001296 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.001479 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.001415 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.102890 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.102853 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.103070 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.102929 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpkqq\" (UniqueName: \"kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.103070 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.102962 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.103070 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:52:03.103017 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4937319b-serving-cert: secret "maas-api-e2e-mt-4937319b-serving-cert" not found Jul 06 14:52:03.103185 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:52:03.103093 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls podName:636dc58e-9a50-4eb2-bef0-143d75df1ee2 nodeName:}" failed. No retries permitted until 2026-07-06 14:52:03.603074062 +0000 UTC m=+1758.374096369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls") pod "maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" (UID: "636dc58e-9a50-4eb2-bef0-143d75df1ee2") : secret "maas-api-e2e-mt-4937319b-serving-cert" not found Jul 06 14:52:03.103744 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.103725 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.118645 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.118614 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpkqq\" (UniqueName: \"kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.608131 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.608093 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.610761 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.610736 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") pod \"maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.816455 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.816425 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:03.968432 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:03.968387 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:03.978998 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:52:03.978955 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636dc58e_9a50_4eb2_bef0_143d75df1ee2.slice/crio-f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff WatchSource:0}: Error finding container f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff: Status 404 returned error can't find the container with id f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff Jul 06 14:52:04.633198 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:04.633098 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" event={"ID":"636dc58e-9a50-4eb2-bef0-143d75df1ee2","Type":"ContainerStarted","Data":"b8f3dfc2c48d447b8ded1210174c3a9f98d95141b751a89f9ca84861e4475a4b"} Jul 06 14:52:04.633198 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:04.633159 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" event={"ID":"636dc58e-9a50-4eb2-bef0-143d75df1ee2","Type":"ContainerStarted","Data":"f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff"} Jul 06 14:52:04.633405 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:04.633282 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:10.644570 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:10.644542 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:10.662287 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:10.662224 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" podStartSLOduration=8.347841115 podStartE2EDuration="8.662103569s" totalImagesPullingTime="314.262454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:52:04.666083903 +0000 UTC m=+1759.437106234" watchObservedRunningTime="2026-07-06 14:52:10.662103569 +0000 UTC m=+1765.433125900" Jul 06 14:52:45.798483 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:45.798457 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:52:45.802713 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:45.802690 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:52:45.808261 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:45.808242 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:52:45.812255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:45.812237 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:52:55.646845 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.646812 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:55.647335 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.647023 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" podUID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" containerName="maas-api" containerID="cri-o://b8f3dfc2c48d447b8ded1210174c3a9f98d95141b751a89f9ca84861e4475a4b" gracePeriod=30 Jul 06 14:52:55.900882 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.900850 2552 generic.go:356] "Generic (PLEG): container finished" podID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" containerID="b8f3dfc2c48d447b8ded1210174c3a9f98d95141b751a89f9ca84861e4475a4b" exitCode=0 Jul 06 14:52:55.901006 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.900929 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" event={"ID":"636dc58e-9a50-4eb2-bef0-143d75df1ee2","Type":"ContainerDied","Data":"b8f3dfc2c48d447b8ded1210174c3a9f98d95141b751a89f9ca84861e4475a4b"} Jul 06 14:52:55.901006 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.900994 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" event={"ID":"636dc58e-9a50-4eb2-bef0-143d75df1ee2","Type":"ContainerDied","Data":"f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff"} Jul 06 14:52:55.901094 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.901011 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2b029748df4d779aa1c6888517a400cc36f356737bb989af4eae93c53cf93ff" Jul 06 14:52:55.914304 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.914281 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:55.944072 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.944037 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") pod \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " Jul 06 14:52:55.944268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.944100 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca\") pod \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " Jul 06 14:52:55.944268 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.944149 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpkqq\" (UniqueName: \"kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq\") pod \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\" (UID: \"636dc58e-9a50-4eb2-bef0-143d75df1ee2\") " Jul 06 14:52:55.944850 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.944718 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "636dc58e-9a50-4eb2-bef0-143d75df1ee2" (UID: "636dc58e-9a50-4eb2-bef0-143d75df1ee2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:52:55.946740 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.946710 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "636dc58e-9a50-4eb2-bef0-143d75df1ee2" (UID: "636dc58e-9a50-4eb2-bef0-143d75df1ee2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:52:55.946850 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:55.946818 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq" (OuterVolumeSpecName: "kube-api-access-kpkqq") pod "636dc58e-9a50-4eb2-bef0-143d75df1ee2" (UID: "636dc58e-9a50-4eb2-bef0-143d75df1ee2"). InnerVolumeSpecName "kube-api-access-kpkqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:52:56.045757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.045726 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/636dc58e-9a50-4eb2-bef0-143d75df1ee2-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:52:56.045757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.045752 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpkqq\" (UniqueName: \"kubernetes.io/projected/636dc58e-9a50-4eb2-bef0-143d75df1ee2-kube-api-access-kpkqq\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:52:56.045757 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.045764 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/636dc58e-9a50-4eb2-bef0-143d75df1ee2-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:52:56.907754 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.907723 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg" Jul 06 14:52:56.947614 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.947576 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:56.952947 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:56.952919 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4937319b-56cd97dd6-lqdrg"] Jul 06 14:52:57.754959 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:52:57.754926 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" path="/var/lib/kubelet/pods/636dc58e-9a50-4eb2-bef0-143d75df1ee2/volumes" Jul 06 14:54:50.687306 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.687262 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:54:50.688137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.687654 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" containerName="maas-api" Jul 06 14:54:50.688137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.687666 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" containerName="maas-api" Jul 06 14:54:50.688137 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.687739 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="636dc58e-9a50-4eb2-bef0-143d75df1ee2" containerName="maas-api" Jul 06 14:54:50.690580 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.690562 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.692954 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.692936 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-ebef4e-serving-cert\"" Jul 06 14:54:50.702962 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.702936 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:54:50.734875 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.734835 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-752s7\" (UniqueName: \"kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.735077 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.734897 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.735077 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.735005 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.836039 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.836001 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.836239 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.836058 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-752s7\" (UniqueName: \"kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.836239 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.836123 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.836239 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:50.836235 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-ebef4e-serving-cert: secret "maas-api-e2e-inf-a-ebef4e-serving-cert" not found Jul 06 14:54:50.836511 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:50.836304 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls podName:a9b7ff93-307f-4f3b-af48-691549670ab5 nodeName:}" failed. No retries permitted until 2026-07-06 14:54:51.336282853 +0000 UTC m=+1926.107305179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls") pod "maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" (UID: "a9b7ff93-307f-4f3b-af48-691549670ab5") : secret "maas-api-e2e-inf-a-ebef4e-serving-cert" not found Jul 06 14:54:50.836756 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.836733 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:50.844820 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:50.844797 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-752s7\" (UniqueName: \"kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:51.341315 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:51.341275 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:51.343759 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:51.343732 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") pod \"maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:51.601427 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:51.601303 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:51.741467 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:51.741437 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:54:51.743762 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:54:51.743727 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b7ff93_307f_4f3b_af48_691549670ab5.slice/crio-f665e3c9c41a49ec27caa0730b80014f48fba0fe0bc37ff1131013baac004587 WatchSource:0}: Error finding container f665e3c9c41a49ec27caa0730b80014f48fba0fe0bc37ff1131013baac004587: Status 404 returned error can't find the container with id f665e3c9c41a49ec27caa0730b80014f48fba0fe0bc37ff1131013baac004587 Jul 06 14:54:51.745171 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:51.745154 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:54:52.505260 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:52.505227 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" event={"ID":"a9b7ff93-307f-4f3b-af48-691549670ab5","Type":"ContainerStarted","Data":"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a"} Jul 06 14:54:52.505486 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:52.505284 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" event={"ID":"a9b7ff93-307f-4f3b-af48-691549670ab5","Type":"ContainerStarted","Data":"f665e3c9c41a49ec27caa0730b80014f48fba0fe0bc37ff1131013baac004587"} Jul 06 14:54:52.505486 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:52.505342 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:52.523538 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:52.523457 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" podStartSLOduration=1.9111204160000002 podStartE2EDuration="2.523439791s" totalImagesPullingTime="612.319375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:52.522168804 +0000 UTC m=+1927.293191139" watchObservedRunningTime="2026-07-06 14:54:52.523439791 +0000 UTC m=+1927.294462120" Jul 06 14:54:58.515360 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:58.515307 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:54:58.997492 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:58.997458 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:54:59.001308 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.001285 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.004034 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.004012 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-139570-serving-cert\"" Jul 06 14:54:59.012590 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.012561 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:54:59.109670 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.109622 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.109670 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.109681 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.109896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.109763 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgrlp\" (UniqueName: \"kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.210953 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.210913 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.211130 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.210960 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.211130 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.211004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgrlp\" (UniqueName: \"kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.211210 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:59.211145 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-139570-serving-cert: secret "maas-api-e2e-inf-b-139570-serving-cert" not found Jul 06 14:54:59.211248 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:59.211234 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls podName:c8240798-99fc-469a-b5f0-dc14ee6fd2d8 nodeName:}" failed. No retries permitted until 2026-07-06 14:54:59.711212495 +0000 UTC m=+1934.482234810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls") pod "maas-api-e2e-inf-b-139570-76c56d68c8-x492t" (UID: "c8240798-99fc-469a-b5f0-dc14ee6fd2d8") : secret "maas-api-e2e-inf-b-139570-serving-cert" not found Jul 06 14:54:59.211603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.211584 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.221255 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.221225 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgrlp\" (UniqueName: \"kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.715802 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:54:59.715761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:54:59.716196 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:59.715920 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-139570-serving-cert: secret "maas-api-e2e-inf-b-139570-serving-cert" not found Jul 06 14:54:59.716196 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:54:59.715989 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls podName:c8240798-99fc-469a-b5f0-dc14ee6fd2d8 nodeName:}" failed. No retries permitted until 2026-07-06 14:55:00.71597292 +0000 UTC m=+1935.486995228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls") pod "maas-api-e2e-inf-b-139570-76c56d68c8-x492t" (UID: "c8240798-99fc-469a-b5f0-dc14ee6fd2d8") : secret "maas-api-e2e-inf-b-139570-serving-cert" not found Jul 06 14:55:00.726713 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:00.726677 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:55:00.729079 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:00.729058 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"maas-api-e2e-inf-b-139570-76c56d68c8-x492t\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:55:00.813582 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:00.813544 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:55:00.954123 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:00.954088 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:55:00.957111 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:55:00.957085 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8240798_99fc_469a_b5f0_dc14ee6fd2d8.slice/crio-77d70ca46be61022defde1a997de5756ea0641485e56046f97e1b24b6e7f429a WatchSource:0}: Error finding container 77d70ca46be61022defde1a997de5756ea0641485e56046f97e1b24b6e7f429a: Status 404 returned error can't find the container with id 77d70ca46be61022defde1a997de5756ea0641485e56046f97e1b24b6e7f429a Jul 06 14:55:01.554726 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:01.554694 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" event={"ID":"c8240798-99fc-469a-b5f0-dc14ee6fd2d8","Type":"ContainerStarted","Data":"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642"} Jul 06 14:55:01.554893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:01.554758 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" event={"ID":"c8240798-99fc-469a-b5f0-dc14ee6fd2d8","Type":"ContainerStarted","Data":"77d70ca46be61022defde1a997de5756ea0641485e56046f97e1b24b6e7f429a"} Jul 06 14:55:01.554893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:01.554800 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:55:01.574132 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:01.574080 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" podStartSLOduration=3.119930801 podStartE2EDuration="3.574062922s" totalImagesPullingTime="454.132121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:55:01.572976536 +0000 UTC m=+1936.343998866" watchObservedRunningTime="2026-07-06 14:55:01.574062922 +0000 UTC m=+1936.345085273" Jul 06 14:55:07.565932 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:55:07.565894 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:56:19.964453 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:19.964419 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:56:19.965006 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:19.964646 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" podUID="a9b7ff93-307f-4f3b-af48-691549670ab5" containerName="maas-api" containerID="cri-o://8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a" gracePeriod=30 Jul 06 14:56:20.215726 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.215663 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:56:20.316516 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.316477 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") pod \"a9b7ff93-307f-4f3b-af48-691549670ab5\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " Jul 06 14:56:20.316691 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.316545 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-752s7\" (UniqueName: \"kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7\") pod \"a9b7ff93-307f-4f3b-af48-691549670ab5\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " Jul 06 14:56:20.316691 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.316583 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca\") pod \"a9b7ff93-307f-4f3b-af48-691549670ab5\" (UID: \"a9b7ff93-307f-4f3b-af48-691549670ab5\") " Jul 06 14:56:20.316938 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.316910 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9b7ff93-307f-4f3b-af48-691549670ab5" (UID: "a9b7ff93-307f-4f3b-af48-691549670ab5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:56:20.318624 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.318598 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9b7ff93-307f-4f3b-af48-691549670ab5" (UID: "a9b7ff93-307f-4f3b-af48-691549670ab5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:56:20.318716 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.318656 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7" (OuterVolumeSpecName: "kube-api-access-752s7") pod "a9b7ff93-307f-4f3b-af48-691549670ab5" (UID: "a9b7ff93-307f-4f3b-af48-691549670ab5"). InnerVolumeSpecName "kube-api-access-752s7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:56:20.418777 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.418113 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b7ff93-307f-4f3b-af48-691549670ab5-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:56:20.418777 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.418165 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-752s7\" (UniqueName: \"kubernetes.io/projected/a9b7ff93-307f-4f3b-af48-691549670ab5-kube-api-access-752s7\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:56:20.418777 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.418180 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b7ff93-307f-4f3b-af48-691549670ab5-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:56:20.990030 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.989993 2552 generic.go:356] "Generic (PLEG): container finished" podID="a9b7ff93-307f-4f3b-af48-691549670ab5" containerID="8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a" exitCode=0 Jul 06 14:56:20.990508 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.990051 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" Jul 06 14:56:20.990508 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.990078 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" event={"ID":"a9b7ff93-307f-4f3b-af48-691549670ab5","Type":"ContainerDied","Data":"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a"} Jul 06 14:56:20.990508 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.990131 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc" event={"ID":"a9b7ff93-307f-4f3b-af48-691549670ab5","Type":"ContainerDied","Data":"f665e3c9c41a49ec27caa0730b80014f48fba0fe0bc37ff1131013baac004587"} Jul 06 14:56:20.990508 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:20.990152 2552 scope.go:117] "RemoveContainer" containerID="8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a" Jul 06 14:56:21.000552 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:21.000536 2552 scope.go:117] "RemoveContainer" containerID="8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a" Jul 06 14:56:21.000851 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:56:21.000831 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a\": container with ID starting with 8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a not found: ID does not exist" containerID="8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a" Jul 06 14:56:21.000902 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:21.000860 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a"} err="failed to get container status \"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a\": rpc error: code = NotFound desc = could not find container \"8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a\": container with ID starting with 8bd9e96a56aa1da1cba3e32ee8e7ac8bc53883b97e9d3e51e9ba5a471fb0001a not found: ID does not exist" Jul 06 14:56:21.013258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:21.013231 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:56:21.017883 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:21.017858 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ebef4e-65fb595945-cclwc"] Jul 06 14:56:21.754408 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:56:21.754369 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b7ff93-307f-4f3b-af48-691549670ab5" path="/var/lib/kubelet/pods/a9b7ff93-307f-4f3b-af48-691549670ab5/volumes" Jul 06 14:57:04.339822 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.339785 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:57:04.340381 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.340083 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" podUID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" containerName="maas-api" containerID="cri-o://68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642" gracePeriod=30 Jul 06 14:57:04.583927 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.583903 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:57:04.691585 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.691474 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") pod \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " Jul 06 14:57:04.691585 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.691552 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca\") pod \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " Jul 06 14:57:04.691823 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.691667 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgrlp\" (UniqueName: \"kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp\") pod \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\" (UID: \"c8240798-99fc-469a-b5f0-dc14ee6fd2d8\") " Jul 06 14:57:04.691930 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.691896 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8240798-99fc-469a-b5f0-dc14ee6fd2d8" (UID: "c8240798-99fc-469a-b5f0-dc14ee6fd2d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:04.693680 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.693643 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp" (OuterVolumeSpecName: "kube-api-access-sgrlp") pod "c8240798-99fc-469a-b5f0-dc14ee6fd2d8" (UID: "c8240798-99fc-469a-b5f0-dc14ee6fd2d8"). InnerVolumeSpecName "kube-api-access-sgrlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:04.693796 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.693706 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8240798-99fc-469a-b5f0-dc14ee6fd2d8" (UID: "c8240798-99fc-469a-b5f0-dc14ee6fd2d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:57:04.793049 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.793011 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:04.793049 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.793043 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:04.793049 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:04.793054 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgrlp\" (UniqueName: \"kubernetes.io/projected/c8240798-99fc-469a-b5f0-dc14ee6fd2d8-kube-api-access-sgrlp\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:05.234764 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.234729 2552 generic.go:356] "Generic (PLEG): container finished" podID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" containerID="68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642" exitCode=0 Jul 06 14:57:05.234973 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.234777 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" Jul 06 14:57:05.234973 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.234816 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" event={"ID":"c8240798-99fc-469a-b5f0-dc14ee6fd2d8","Type":"ContainerDied","Data":"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642"} Jul 06 14:57:05.234973 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.234875 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t" event={"ID":"c8240798-99fc-469a-b5f0-dc14ee6fd2d8","Type":"ContainerDied","Data":"77d70ca46be61022defde1a997de5756ea0641485e56046f97e1b24b6e7f429a"} Jul 06 14:57:05.234973 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.234904 2552 scope.go:117] "RemoveContainer" containerID="68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642" Jul 06 14:57:05.244237 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.244218 2552 scope.go:117] "RemoveContainer" containerID="68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642" Jul 06 14:57:05.244516 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:57:05.244499 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642\": container with ID starting with 68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642 not found: ID does not exist" containerID="68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642" Jul 06 14:57:05.244564 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.244526 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642"} err="failed to get container status \"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642\": rpc error: code = NotFound desc = could not find container \"68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642\": container with ID starting with 68b559210eb151b09c27d7b464ca848766764b50d4f0347c6cea72fadf56e642 not found: ID does not exist" Jul 06 14:57:05.263548 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.263517 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:57:05.267555 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.267532 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-139570-76c56d68c8-x492t"] Jul 06 14:57:05.754075 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:05.754044 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" path="/var/lib/kubelet/pods/c8240798-99fc-469a-b5f0-dc14ee6fd2d8/volumes" Jul 06 14:57:15.597720 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.597633 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:15.598193 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598177 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b7ff93-307f-4f3b-af48-691549670ab5" containerName="maas-api" Jul 06 14:57:15.598238 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598196 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b7ff93-307f-4f3b-af48-691549670ab5" containerName="maas-api" Jul 06 14:57:15.598238 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598226 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" containerName="maas-api" Jul 06 14:57:15.598302 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598235 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" containerName="maas-api" Jul 06 14:57:15.598360 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598347 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8240798-99fc-469a-b5f0-dc14ee6fd2d8" containerName="maas-api" Jul 06 14:57:15.598400 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.598362 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b7ff93-307f-4f3b-af48-691549670ab5" containerName="maas-api" Jul 06 14:57:15.602893 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.602870 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.604890 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.604871 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-7ed418-serving-cert\"" Jul 06 14:57:15.613441 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.613414 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:15.693289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.693251 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.693289 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.693296 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.693535 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.693426 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sv5h\" (UniqueName: \"kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.794159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.794118 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sv5h\" (UniqueName: \"kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.794159 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.794163 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.794487 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.794191 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.794965 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.794946 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.796824 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.796801 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.805223 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.805197 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sv5h\" (UniqueName: \"kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h\") pod \"maas-api-e2e-api-a-7ed418-7b858cd88-njns5\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:15.914333 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:15.914233 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:16.056259 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:16.056228 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:16.057761 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:57:16.057737 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297cd4f5_774b_476b_a97d_198e3582336b.slice/crio-201fca47716affab37727f0952d04bdd5edab31512b7d1cd6c0927d6a62e3b6a WatchSource:0}: Error finding container 201fca47716affab37727f0952d04bdd5edab31512b7d1cd6c0927d6a62e3b6a: Status 404 returned error can't find the container with id 201fca47716affab37727f0952d04bdd5edab31512b7d1cd6c0927d6a62e3b6a Jul 06 14:57:16.294901 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:16.294870 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" event={"ID":"297cd4f5-774b-476b-a97d-198e3582336b","Type":"ContainerStarted","Data":"201fca47716affab37727f0952d04bdd5edab31512b7d1cd6c0927d6a62e3b6a"} Jul 06 14:57:17.302170 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:17.302128 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" event={"ID":"297cd4f5-774b-476b-a97d-198e3582336b","Type":"ContainerStarted","Data":"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18"} Jul 06 14:57:17.302648 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:17.302253 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:17.324189 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:17.324133 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" podStartSLOduration=1.827506796 podStartE2EDuration="2.324116034s" totalImagesPullingTime="496.609238ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:57:17.321060278 +0000 UTC m=+2072.092082607" watchObservedRunningTime="2026-07-06 14:57:17.324116034 +0000 UTC m=+2072.095138363" Jul 06 14:57:23.313763 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:23.313731 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:23.979084 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:23.979051 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:23.982606 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:23.982586 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:23.988054 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:23.988029 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-ed33e6-serving-cert\"" Jul 06 14:57:24.008811 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.008772 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:24.059532 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.059497 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.059704 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.059551 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsp4r\" (UniqueName: \"kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.059704 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.059629 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.160682 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.160644 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.160864 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.160705 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsp4r\" (UniqueName: \"kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.160864 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.160728 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.160864 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:57:24.160844 2552 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-ed33e6-serving-cert: secret "maas-api-e2e-api-b-ed33e6-serving-cert" not found Jul 06 14:57:24.160978 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:57:24.160900 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls podName:8ff12462-5df5-48c4-9208-1805e5a77d5f nodeName:}" failed. No retries permitted until 2026-07-06 14:57:24.660881636 +0000 UTC m=+2079.431903944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls") pod "maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" (UID: "8ff12462-5df5-48c4-9208-1805e5a77d5f") : secret "maas-api-e2e-api-b-ed33e6-serving-cert" not found Jul 06 14:57:24.161412 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.161391 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.178770 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.178740 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsp4r\" (UniqueName: \"kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.665429 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.665393 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.667780 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.667757 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") pod \"maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:24.892781 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:24.892737 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:25.038759 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:25.038732 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:25.041398 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:57:25.041365 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff12462_5df5_48c4_9208_1805e5a77d5f.slice/crio-789e4de1fbb8eada11721c9c8fb9f6061b4705f8bf318f917b7ebe7b548a202e WatchSource:0}: Error finding container 789e4de1fbb8eada11721c9c8fb9f6061b4705f8bf318f917b7ebe7b548a202e: Status 404 returned error can't find the container with id 789e4de1fbb8eada11721c9c8fb9f6061b4705f8bf318f917b7ebe7b548a202e Jul 06 14:57:25.352172 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:25.352130 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" event={"ID":"8ff12462-5df5-48c4-9208-1805e5a77d5f","Type":"ContainerStarted","Data":"789e4de1fbb8eada11721c9c8fb9f6061b4705f8bf318f917b7ebe7b548a202e"} Jul 06 14:57:26.358533 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:26.358497 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" event={"ID":"8ff12462-5df5-48c4-9208-1805e5a77d5f","Type":"ContainerStarted","Data":"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc"} Jul 06 14:57:26.358938 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:26.358599 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:26.380421 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:26.380361 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" podStartSLOduration=3.051134223 podStartE2EDuration="3.380343815s" totalImagesPullingTime="329.209592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:57:26.379109722 +0000 UTC m=+2081.150132052" watchObservedRunningTime="2026-07-06 14:57:26.380343815 +0000 UTC m=+2081.151366147" Jul 06 14:57:32.370029 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:32.369997 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:37.364704 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.364671 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:37.365157 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.364876 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" podUID="297cd4f5-774b-476b-a97d-198e3582336b" containerName="maas-api" containerID="cri-o://d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18" gracePeriod=30 Jul 06 14:57:37.606031 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.606007 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:37.678931 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.678831 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca\") pod \"297cd4f5-774b-476b-a97d-198e3582336b\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " Jul 06 14:57:37.678931 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.678871 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls\") pod \"297cd4f5-774b-476b-a97d-198e3582336b\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " Jul 06 14:57:37.679161 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.678946 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sv5h\" (UniqueName: \"kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h\") pod \"297cd4f5-774b-476b-a97d-198e3582336b\" (UID: \"297cd4f5-774b-476b-a97d-198e3582336b\") " Jul 06 14:57:37.679258 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.679229 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "297cd4f5-774b-476b-a97d-198e3582336b" (UID: "297cd4f5-774b-476b-a97d-198e3582336b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:37.681239 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.681209 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h" (OuterVolumeSpecName: "kube-api-access-7sv5h") pod "297cd4f5-774b-476b-a97d-198e3582336b" (UID: "297cd4f5-774b-476b-a97d-198e3582336b"). InnerVolumeSpecName "kube-api-access-7sv5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:37.681389 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.681272 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "297cd4f5-774b-476b-a97d-198e3582336b" (UID: "297cd4f5-774b-476b-a97d-198e3582336b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:57:37.780084 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.780046 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sv5h\" (UniqueName: \"kubernetes.io/projected/297cd4f5-774b-476b-a97d-198e3582336b-kube-api-access-7sv5h\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:37.780084 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.780081 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/297cd4f5-774b-476b-a97d-198e3582336b-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:37.780084 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:37.780092 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/297cd4f5-774b-476b-a97d-198e3582336b-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:38.429896 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.429858 2552 generic.go:356] "Generic (PLEG): container finished" podID="297cd4f5-774b-476b-a97d-198e3582336b" containerID="d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18" exitCode=0 Jul 06 14:57:38.430349 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.429911 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" Jul 06 14:57:38.430349 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.429955 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" event={"ID":"297cd4f5-774b-476b-a97d-198e3582336b","Type":"ContainerDied","Data":"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18"} Jul 06 14:57:38.430349 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.430007 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5" event={"ID":"297cd4f5-774b-476b-a97d-198e3582336b","Type":"ContainerDied","Data":"201fca47716affab37727f0952d04bdd5edab31512b7d1cd6c0927d6a62e3b6a"} Jul 06 14:57:38.430349 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.430029 2552 scope.go:117] "RemoveContainer" containerID="d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18" Jul 06 14:57:38.438952 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.438934 2552 scope.go:117] "RemoveContainer" containerID="d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18" Jul 06 14:57:38.439213 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:57:38.439196 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18\": container with ID starting with d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18 not found: ID does not exist" containerID="d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18" Jul 06 14:57:38.439266 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.439223 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18"} err="failed to get container status \"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18\": rpc error: code = NotFound desc = could not find container \"d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18\": container with ID starting with d1d2374d13bc6bc03d9b1710a46afbe28b1992eef56219d22c192e8a60274d18 not found: ID does not exist" Jul 06 14:57:38.457472 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.457441 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:38.468451 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:38.468426 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-7ed418-7b858cd88-njns5"] Jul 06 14:57:39.754274 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:39.754238 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="297cd4f5-774b-476b-a97d-198e3582336b" path="/var/lib/kubelet/pods/297cd4f5-774b-476b-a97d-198e3582336b/volumes" Jul 06 14:57:44.787855 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:44.787812 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:44.788458 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:44.788084 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" podUID="8ff12462-5df5-48c4-9208-1805e5a77d5f" containerName="maas-api" containerID="cri-o://27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc" gracePeriod=30 Jul 06 14:57:45.058021 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.057995 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:45.144603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.144571 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca\") pod \"8ff12462-5df5-48c4-9208-1805e5a77d5f\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " Jul 06 14:57:45.144603 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.144607 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsp4r\" (UniqueName: \"kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r\") pod \"8ff12462-5df5-48c4-9208-1805e5a77d5f\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " Jul 06 14:57:45.144868 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.144624 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") pod \"8ff12462-5df5-48c4-9208-1805e5a77d5f\" (UID: \"8ff12462-5df5-48c4-9208-1805e5a77d5f\") " Jul 06 14:57:45.145041 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.145013 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ff12462-5df5-48c4-9208-1805e5a77d5f" (UID: "8ff12462-5df5-48c4-9208-1805e5a77d5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:45.146709 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.146685 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ff12462-5df5-48c4-9208-1805e5a77d5f" (UID: "8ff12462-5df5-48c4-9208-1805e5a77d5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:57:45.146805 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.146785 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r" (OuterVolumeSpecName: "kube-api-access-bsp4r") pod "8ff12462-5df5-48c4-9208-1805e5a77d5f" (UID: "8ff12462-5df5-48c4-9208-1805e5a77d5f"). InnerVolumeSpecName "kube-api-access-bsp4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:45.246126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.246079 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ff12462-5df5-48c4-9208-1805e5a77d5f-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.246126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.246114 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsp4r\" (UniqueName: \"kubernetes.io/projected/8ff12462-5df5-48c4-9208-1805e5a77d5f-kube-api-access-bsp4r\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.246126 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.246125 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ff12462-5df5-48c4-9208-1805e5a77d5f-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.468158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.468062 2552 generic.go:356] "Generic (PLEG): container finished" podID="8ff12462-5df5-48c4-9208-1805e5a77d5f" containerID="27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc" exitCode=0 Jul 06 14:57:45.468158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.468107 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" Jul 06 14:57:45.468158 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.468146 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" event={"ID":"8ff12462-5df5-48c4-9208-1805e5a77d5f","Type":"ContainerDied","Data":"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc"} Jul 06 14:57:45.468473 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.468200 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7" event={"ID":"8ff12462-5df5-48c4-9208-1805e5a77d5f","Type":"ContainerDied","Data":"789e4de1fbb8eada11721c9c8fb9f6061b4705f8bf318f917b7ebe7b548a202e"} Jul 06 14:57:45.468473 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.468223 2552 scope.go:117] "RemoveContainer" containerID="27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc" Jul 06 14:57:45.477558 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.477537 2552 scope.go:117] "RemoveContainer" containerID="27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc" Jul 06 14:57:45.477817 ip-10-0-141-213 kubenswrapper[2552]: E0706 14:57:45.477797 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc\": container with ID starting with 27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc not found: ID does not exist" containerID="27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc" Jul 06 14:57:45.477897 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.477825 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc"} err="failed to get container status \"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc\": rpc error: code = NotFound desc = could not find container \"27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc\": container with ID starting with 27d8c6ff0fc6cfb2eb6f20381fbf77245193f08c8dbff5a020f19914cbdd29bc not found: ID does not exist" Jul 06 14:57:45.491519 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.491488 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:45.496552 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.496530 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-ed33e6-74787467bb-kn6s7"] Jul 06 14:57:45.760703 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.760627 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff12462-5df5-48c4-9208-1805e5a77d5f" path="/var/lib/kubelet/pods/8ff12462-5df5-48c4-9208-1805e5a77d5f/volumes" Jul 06 14:57:45.833036 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.832926 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:57:45.846125 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.837834 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:57:45.846125 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.842888 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 14:57:45.847399 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:45.847382 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 14:57:54.961437 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961397 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 14:57:54.961823 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961743 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297cd4f5-774b-476b-a97d-198e3582336b" containerName="maas-api" Jul 06 14:57:54.961823 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961754 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="297cd4f5-774b-476b-a97d-198e3582336b" containerName="maas-api" Jul 06 14:57:54.961823 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961776 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff12462-5df5-48c4-9208-1805e5a77d5f" containerName="maas-api" Jul 06 14:57:54.961823 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961781 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff12462-5df5-48c4-9208-1805e5a77d5f" containerName="maas-api" Jul 06 14:57:54.961971 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961842 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="297cd4f5-774b-476b-a97d-198e3582336b" containerName="maas-api" Jul 06 14:57:54.961971 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.961851 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff12462-5df5-48c4-9208-1805e5a77d5f" containerName="maas-api" Jul 06 14:57:54.964762 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.964744 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:54.978738 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:54.978713 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-58d9da-serving-cert\"" Jul 06 14:57:55.013445 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.013406 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 14:57:55.038363 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.038304 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.038531 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.038427 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.038531 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.038455 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5zzh\" (UniqueName: \"kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.139670 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.139631 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.139874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.139695 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.139874 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.139714 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5zzh\" (UniqueName: \"kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.140454 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.140431 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.142094 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.142071 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.151463 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.151440 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5zzh\" (UniqueName: \"kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh\") pod \"maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.275203 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.275115 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:55.438251 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:57:55.438210 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13dbe7d8_d8dd_4a50_b081_ebd0e297e10a.slice/crio-ddde5c44edd339159210053b685166e24301bc8edec3a63ddf6026aa3a5783bf WatchSource:0}: Error finding container ddde5c44edd339159210053b685166e24301bc8edec3a63ddf6026aa3a5783bf: Status 404 returned error can't find the container with id ddde5c44edd339159210053b685166e24301bc8edec3a63ddf6026aa3a5783bf Jul 06 14:57:55.450116 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.450090 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 14:57:55.518655 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:55.518620 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" event={"ID":"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a","Type":"ContainerStarted","Data":"ddde5c44edd339159210053b685166e24301bc8edec3a63ddf6026aa3a5783bf"} Jul 06 14:57:56.525232 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:56.525188 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" event={"ID":"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a","Type":"ContainerStarted","Data":"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf"} Jul 06 14:57:56.525745 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:56.525356 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:57:56.547704 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:57:56.547655 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" podStartSLOduration=2.158648139 podStartE2EDuration="2.547637388s" totalImagesPullingTime="388.989249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:57:56.545459886 +0000 UTC m=+2111.316482215" watchObservedRunningTime="2026-07-06 14:57:56.547637388 +0000 UTC m=+2111.318659718" Jul 06 14:58:02.536738 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:02.536709 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 14:58:03.224016 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.223978 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 14:58:03.228142 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.228110 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.230627 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.230604 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-a38db2-serving-cert\"" Jul 06 14:58:03.245887 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.245857 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 14:58:03.311749 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.311706 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.311922 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.311771 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.311922 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.311820 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6bgk\" (UniqueName: \"kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.412486 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.412441 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.412660 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.412509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.412660 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.412533 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6bgk\" (UniqueName: \"kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.413247 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.413223 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.414997 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.414977 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.424580 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.424552 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6bgk\" (UniqueName: \"kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk\") pod \"maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.539213 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.539182 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:03.681190 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:03.681163 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 14:58:03.683334 ip-10-0-141-213 kubenswrapper[2552]: W0706 14:58:03.683288 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35008dfe_ff1f_49e2_a009_39ff62464b84.slice/crio-65cab4ee7fb97a8dfcfd60d8539d8c55b87611ed7ee418d47186442bb4478790 WatchSource:0}: Error finding container 65cab4ee7fb97a8dfcfd60d8539d8c55b87611ed7ee418d47186442bb4478790: Status 404 returned error can't find the container with id 65cab4ee7fb97a8dfcfd60d8539d8c55b87611ed7ee418d47186442bb4478790 Jul 06 14:58:04.567675 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:04.567637 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" event={"ID":"35008dfe-ff1f-49e2-a009-39ff62464b84","Type":"ContainerStarted","Data":"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344"} Jul 06 14:58:04.568152 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:04.567694 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" event={"ID":"35008dfe-ff1f-49e2-a009-39ff62464b84","Type":"ContainerStarted","Data":"65cab4ee7fb97a8dfcfd60d8539d8c55b87611ed7ee418d47186442bb4478790"} Jul 06 14:58:04.568152 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:04.567769 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:04.583717 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:04.583662 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" podStartSLOduration=1.253157844 podStartE2EDuration="1.583646372s" totalImagesPullingTime="330.488528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:04.582848225 +0000 UTC m=+2119.353870591" watchObservedRunningTime="2026-07-06 14:58:04.583646372 +0000 UTC m=+2119.354668748" Jul 06 14:58:10.577694 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:10.577667 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 14:58:45.940236 ip-10-0-141-213 kubenswrapper[2552]: I0706 14:58:45.940153 2552 scope.go:117] "RemoveContainer" containerID="b8f3dfc2c48d447b8ded1210174c3a9f98d95141b751a89f9ca84861e4475a4b" Jul 06 15:02:45.867077 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:02:45.866958 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 15:02:45.871625 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:02:45.871606 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 15:02:45.879182 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:02:45.879160 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 15:02:45.883459 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:02:45.883442 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ftg5_23e7f036-b2b0-499a-a66e-fa1c90fc05dc/ovn-acl-logging/0.log" Jul 06 15:03:44.282645 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.282608 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 15:03:44.283103 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.282849 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" podUID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" containerName="maas-api" containerID="cri-o://05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf" gracePeriod=30 Jul 06 15:03:44.529196 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.529171 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 15:03:44.598025 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.597989 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca\") pod \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " Jul 06 15:03:44.598380 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.598356 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" (UID: "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:03:44.698983 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.698940 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls\") pod \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " Jul 06 15:03:44.698983 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.698985 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5zzh\" (UniqueName: \"kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh\") pod \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\" (UID: \"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a\") " Jul 06 15:03:44.699256 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.699240 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:03:44.701060 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.701028 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" (UID: "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:03:44.701060 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.701048 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh" (OuterVolumeSpecName: "kube-api-access-k5zzh") pod "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" (UID: "13dbe7d8-d8dd-4a50-b081-ebd0e297e10a"). InnerVolumeSpecName "kube-api-access-k5zzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:03:44.799746 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.799708 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:03:44.799746 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:44.799739 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5zzh\" (UniqueName: \"kubernetes.io/projected/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a-kube-api-access-k5zzh\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:03:45.350982 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.350944 2552 generic.go:356] "Generic (PLEG): container finished" podID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" containerID="05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf" exitCode=0 Jul 06 15:03:45.350982 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.350988 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" Jul 06 15:03:45.351511 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.351028 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" event={"ID":"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a","Type":"ContainerDied","Data":"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf"} Jul 06 15:03:45.351511 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.351074 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl" event={"ID":"13dbe7d8-d8dd-4a50-b081-ebd0e297e10a","Type":"ContainerDied","Data":"ddde5c44edd339159210053b685166e24301bc8edec3a63ddf6026aa3a5783bf"} Jul 06 15:03:45.351511 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.351096 2552 scope.go:117] "RemoveContainer" containerID="05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf" Jul 06 15:03:45.360316 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.360297 2552 scope.go:117] "RemoveContainer" containerID="05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf" Jul 06 15:03:45.360590 ip-10-0-141-213 kubenswrapper[2552]: E0706 15:03:45.360573 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf\": container with ID starting with 05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf not found: ID does not exist" containerID="05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf" Jul 06 15:03:45.360630 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.360600 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf"} err="failed to get container status \"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf\": rpc error: code = NotFound desc = could not find container \"05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf\": container with ID starting with 05a440b5c72bff21800f7dfe16f302998b74c59ecb2bfec6d13e6da9803e6fbf not found: ID does not exist" Jul 06 15:03:45.372371 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.372340 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 15:03:45.381007 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.380983 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-58d9da-86b594486f-kwtxl"] Jul 06 15:03:45.755184 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:03:45.755103 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" path="/var/lib/kubelet/pods/13dbe7d8-d8dd-4a50-b081-ebd0e297e10a/volumes" Jul 06 15:04:09.141286 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.141249 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 15:04:09.141907 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.141532 2552 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" podUID="35008dfe-ff1f-49e2-a009-39ff62464b84" containerName="maas-api" containerID="cri-o://add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344" gracePeriod=30 Jul 06 15:04:09.391396 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.391316 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 15:04:09.475233 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.475203 2552 generic.go:356] "Generic (PLEG): container finished" podID="35008dfe-ff1f-49e2-a009-39ff62464b84" containerID="add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344" exitCode=0 Jul 06 15:04:09.475459 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.475252 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" Jul 06 15:04:09.475459 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.475298 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" event={"ID":"35008dfe-ff1f-49e2-a009-39ff62464b84","Type":"ContainerDied","Data":"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344"} Jul 06 15:04:09.475459 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.475373 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk" event={"ID":"35008dfe-ff1f-49e2-a009-39ff62464b84","Type":"ContainerDied","Data":"65cab4ee7fb97a8dfcfd60d8539d8c55b87611ed7ee418d47186442bb4478790"} Jul 06 15:04:09.475459 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.475410 2552 scope.go:117] "RemoveContainer" containerID="add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344" Jul 06 15:04:09.483827 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.483810 2552 scope.go:117] "RemoveContainer" containerID="add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344" Jul 06 15:04:09.484123 ip-10-0-141-213 kubenswrapper[2552]: E0706 15:04:09.484105 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344\": container with ID starting with add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344 not found: ID does not exist" containerID="add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344" Jul 06 15:04:09.484166 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.484134 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344"} err="failed to get container status \"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344\": rpc error: code = NotFound desc = could not find container \"add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344\": container with ID starting with add246ee2fffab477f8db6ae7f12e4f112a66213bc96483cc5a0c30443144344 not found: ID does not exist" Jul 06 15:04:09.507457 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.507423 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca\") pod \"35008dfe-ff1f-49e2-a009-39ff62464b84\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " Jul 06 15:04:09.507553 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.507492 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls\") pod \"35008dfe-ff1f-49e2-a009-39ff62464b84\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " Jul 06 15:04:09.507553 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.507528 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6bgk\" (UniqueName: \"kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk\") pod \"35008dfe-ff1f-49e2-a009-39ff62464b84\" (UID: \"35008dfe-ff1f-49e2-a009-39ff62464b84\") " Jul 06 15:04:09.507885 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.507861 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35008dfe-ff1f-49e2-a009-39ff62464b84" (UID: "35008dfe-ff1f-49e2-a009-39ff62464b84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 15:04:09.509561 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.509537 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35008dfe-ff1f-49e2-a009-39ff62464b84" (UID: "35008dfe-ff1f-49e2-a009-39ff62464b84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 15:04:09.509641 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.509602 2552 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk" (OuterVolumeSpecName: "kube-api-access-t6bgk") pod "35008dfe-ff1f-49e2-a009-39ff62464b84" (UID: "35008dfe-ff1f-49e2-a009-39ff62464b84"). InnerVolumeSpecName "kube-api-access-t6bgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 15:04:09.608175 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.608144 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35008dfe-ff1f-49e2-a009-39ff62464b84-openshift-service-ca\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:04:09.608175 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.608172 2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35008dfe-ff1f-49e2-a009-39ff62464b84-maas-api-tls\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:04:09.608175 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.608181 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6bgk\" (UniqueName: \"kubernetes.io/projected/35008dfe-ff1f-49e2-a009-39ff62464b84-kube-api-access-t6bgk\") on node \"ip-10-0-141-213.ec2.internal\" DevicePath \"\"" Jul 06 15:04:09.797216 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.797178 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 15:04:09.802708 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:09.802680 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a38db2-78664db8d7-l4gxk"] Jul 06 15:04:11.754396 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:11.754364 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35008dfe-ff1f-49e2-a009-39ff62464b84" path="/var/lib/kubelet/pods/35008dfe-ff1f-49e2-a009-39ff62464b84/volumes" Jul 06 15:04:15.959531 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:15.959492 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d9cfb94f-28gth_81862d64-2332-47e8-9cf1-212936fc1ab2/authorino/0.log" Jul 06 15:04:20.944092 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:20.944060 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ff654ddb6-74kvm_79276d4e-dd7e-4e87-b7f9-764025f69455/maas-api/0.log" Jul 06 15:04:21.301017 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:21.300986 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74787c7b7f-rd2ht_5c7036fb-2724-407b-80cb-69816c1a17cc/manager/0.log" Jul 06 15:04:21.425455 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:21.425425 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xk4x2_666c79f7-e3be-4bbc-9cd5-13fe98bb2985/server/0.log" Jul 06 15:04:21.551969 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:21.551891 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gnznn_e8dfbec5-bf40-4b90-b72b-942679b69c68/manager/0.log" Jul 06 15:04:21.829640 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:21.829560 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-557ddcc9b5-vttbb_36bf9105-a209-4b24-b346-85fe501c784e/copy-manifests/0.log" Jul 06 15:04:21.837373 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:21.837349 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-557ddcc9b5-vttbb_36bf9105-a209-4b24-b346-85fe501c784e/manager/0.log" Jul 06 15:04:22.812202 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.812174 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/util/0.log" Jul 06 15:04:22.819700 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.819677 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/pull/0.log" Jul 06 15:04:22.831050 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.831019 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/extract/0.log" Jul 06 15:04:22.947686 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.947653 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/util/0.log" Jul 06 15:04:22.955367 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.955317 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/pull/0.log" Jul 06 15:04:22.962651 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:22.962633 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/extract/0.log" Jul 06 15:04:23.069486 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.069413 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/util/0.log" Jul 06 15:04:23.076739 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.076711 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/pull/0.log" Jul 06 15:04:23.085003 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.084978 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/extract/0.log" Jul 06 15:04:23.195292 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.195266 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/util/0.log" Jul 06 15:04:23.205172 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.205153 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/pull/0.log" Jul 06 15:04:23.214662 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.214631 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/extract/0.log" Jul 06 15:04:23.333165 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.333077 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d9cfb94f-28gth_81862d64-2332-47e8-9cf1-212936fc1ab2/authorino/0.log" Jul 06 15:04:23.672055 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:23.671973 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hg278_6da2126a-d27a-454e-82ec-7f64e3615fe3/kuadrant-console-plugin/0.log" Jul 06 15:04:24.806813 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:24.806788 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b5f845fbc-bvglr_cb74fabc-8555-437c-8959-7f50d3b3553e/kube-auth-proxy/0.log" Jul 06 15:04:25.042455 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:25.042423 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9fkt2_f2560319-f9b2-49e8-8a42-13c6308d9fd2/payload-pre-processing/0.log" Jul 06 15:04:25.608695 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:25.608668 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-t6v7r_6b413d22-08df-4f3d-be47-81a5cb8e83ec/main/0.log" Jul 06 15:04:25.961896 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:25.961822 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbsl2hc_ff3d62d3-d65f-4ebd-acd2-68eb1625356a/main/0.log" Jul 06 15:04:34.767839 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:34.767807 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7q979_b1f8d040-e6ef-4596-a93e-b22ff5976566/global-pull-secret-syncer/0.log" Jul 06 15:04:34.960499 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:34.960468 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fcbgm_11f39976-8fee-49d6-9e1e-6d27d2186e6e/konnectivity-agent/0.log" Jul 06 15:04:35.065691 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:35.065657 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-213.ec2.internal_9fd54470ed1e6a3a6cacd405db1be8dc/haproxy/0.log" Jul 06 15:04:39.351375 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.351273 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/extract/0.log" Jul 06 15:04:39.385999 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.385966 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/util/0.log" Jul 06 15:04:39.420175 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.420146 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759twcjc_1bba9bba-b551-49d1-8802-b407209cd504/pull/0.log" Jul 06 15:04:39.450889 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.450853 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/extract/0.log" Jul 06 15:04:39.478754 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.478726 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/util/0.log" Jul 06 15:04:39.504008 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.503978 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tkdj8_764666f5-88ad-4486-841e-4e6bed0c2692/pull/0.log" Jul 06 15:04:39.534484 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.534454 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/extract/0.log" Jul 06 15:04:39.557158 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.557126 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/util/0.log" Jul 06 15:04:39.579983 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.579955 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zllrv_11cb9813-4429-4377-821a-232f74f758ad/pull/0.log" Jul 06 15:04:39.606844 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.606771 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/extract/0.log" Jul 06 15:04:39.631728 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.631701 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/util/0.log" Jul 06 15:04:39.658513 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.658475 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zx2j4_851941b8-0c97-47eb-8011-907044958f5f/pull/0.log" Jul 06 15:04:39.939160 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:39.939075 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d9cfb94f-28gth_81862d64-2332-47e8-9cf1-212936fc1ab2/authorino/0.log" Jul 06 15:04:40.046178 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:40.046150 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hg278_6da2126a-d27a-454e-82ec-7f64e3615fe3/kuadrant-console-plugin/0.log" Jul 06 15:04:42.464442 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:42.464414 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8fdr_b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe/node-exporter/0.log" Jul 06 15:04:42.497470 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:42.497439 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8fdr_b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe/kube-rbac-proxy/0.log" Jul 06 15:04:42.524601 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:42.524578 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f8fdr_b4fba76a-fb9b-4a4c-a2f5-74e55c217dbe/init-textfile/0.log" Jul 06 15:04:44.647250 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647216 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf"] Jul 06 15:04:44.647656 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647624 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" containerName="maas-api" Jul 06 15:04:44.647656 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647638 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" containerName="maas-api" Jul 06 15:04:44.647656 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647648 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35008dfe-ff1f-49e2-a009-39ff62464b84" containerName="maas-api" Jul 06 15:04:44.647656 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647654 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="35008dfe-ff1f-49e2-a009-39ff62464b84" containerName="maas-api" Jul 06 15:04:44.647787 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647731 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13dbe7d8-d8dd-4a50-b081-ebd0e297e10a" containerName="maas-api" Jul 06 15:04:44.647787 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.647740 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="35008dfe-ff1f-49e2-a009-39ff62464b84" containerName="maas-api" Jul 06 15:04:44.658847 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.658816 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.659382 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.659349 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf"] Jul 06 15:04:44.661579 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.661544 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gbbmr\"/\"default-dockercfg-5wsdq\"" Jul 06 15:04:44.661805 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.661784 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbbmr\"/\"openshift-service-ca.crt\"" Jul 06 15:04:44.662297 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.662280 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbbmr\"/\"kube-root-ca.crt\"" Jul 06 15:04:44.715805 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.715760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-lib-modules\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.715805 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.715803 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqkp\" (UniqueName: \"kubernetes.io/projected/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-kube-api-access-ldqkp\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.716019 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.715824 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-podres\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.716019 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.715856 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-sys\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.716019 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.715884 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-proc\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.721105 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.721087 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-srx4c_98931d88-0dc5-4b00-8a2a-01ec2d019c90/networking-console-plugin/0.log" Jul 06 15:04:44.816815 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816782 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-proc\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.816992 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816857 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-lib-modules\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.816992 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816891 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqkp\" (UniqueName: \"kubernetes.io/projected/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-kube-api-access-ldqkp\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.816992 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816902 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-proc\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.816992 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816915 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-podres\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.816992 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.816967 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-sys\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.817189 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.817041 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-sys\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.817189 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.817045 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-lib-modules\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.817189 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.817050 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-podres\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.825125 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.825102 2552 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqkp\" (UniqueName: \"kubernetes.io/projected/9e4f94e1-de7d-4c3e-bda6-0f2daff7a655-kube-api-access-ldqkp\") pod \"perf-node-gather-daemonset-7nzkf\" (UID: \"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655\") " pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:44.972452 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:44.972363 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:45.108270 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.108240 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf"] Jul 06 15:04:45.110498 ip-10-0-141-213 kubenswrapper[2552]: W0706 15:04:45.110458 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9e4f94e1_de7d_4c3e_bda6_0f2daff7a655.slice/crio-0ca81ea9396566321fa6fead1cbcdc44e69c5ad020961c778e24872f046de36e WatchSource:0}: Error finding container 0ca81ea9396566321fa6fead1cbcdc44e69c5ad020961c778e24872f046de36e: Status 404 returned error can't find the container with id 0ca81ea9396566321fa6fead1cbcdc44e69c5ad020961c778e24872f046de36e Jul 06 15:04:45.431499 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.431456 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/1.log" Jul 06 15:04:45.435765 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.435744 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bkj22_62f58d1e-b920-435c-8d01-177a0e80937f/console-operator/2.log" Jul 06 15:04:45.667473 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.667437 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" event={"ID":"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655","Type":"ContainerStarted","Data":"a1d1e1e61eb7f163640b99ab406e90254a95848e79e13bc1311b78733a560c5f"} Jul 06 15:04:45.667872 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.667507 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" event={"ID":"9e4f94e1-de7d-4c3e-bda6-0f2daff7a655","Type":"ContainerStarted","Data":"0ca81ea9396566321fa6fead1cbcdc44e69c5ad020961c778e24872f046de36e"} Jul 06 15:04:45.667872 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.667606 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:45.688200 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.688149 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" podStartSLOduration=1.688131555 podStartE2EDuration="1.688131555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:04:45.685733811 +0000 UTC m=+2520.456756142" watchObservedRunningTime="2026-07-06 15:04:45.688131555 +0000 UTC m=+2520.459153886" Jul 06 15:04:45.934227 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:45.934149 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-ldhwf_8298dda6-ef7e-4962-972f-64264e2a3aff/download-server/0.log" Jul 06 15:04:47.595722 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:47.595696 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9k4wk_16a623a6-4e37-4653-aace-3b63b801e731/dns/0.log" Jul 06 15:04:47.622603 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:47.622576 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9k4wk_16a623a6-4e37-4653-aace-3b63b801e731/kube-rbac-proxy/0.log" Jul 06 15:04:47.764453 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:47.764424 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kwqqr_3cb3e28a-fffd-47d3-b379-e3bed6b8b03f/dns-node-resolver/0.log" Jul 06 15:04:48.363362 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:48.363333 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dx8bm_c35f45ce-4d40-4bb9-b18e-e6eb9917a1db/node-ca/0.log" Jul 06 15:04:49.495704 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:49.495670 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b5f845fbc-bvglr_cb74fabc-8555-437c-8959-7f50d3b3553e/kube-auth-proxy/0.log" Jul 06 15:04:49.595343 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:49.595293 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9fkt2_f2560319-f9b2-49e8-8a42-13c6308d9fd2/payload-pre-processing/0.log" Jul 06 15:04:50.341176 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:50.341152 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4gspr_931ac551-f06e-4543-ad74-05080b31bee7/serve-healthcheck-canary/0.log" Jul 06 15:04:50.899179 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:50.899146 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-272kw_bede46ad-fef0-4794-8d5f-ffd886f610bf/insights-operator/0.log" Jul 06 15:04:50.899577 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:50.899402 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-272kw_bede46ad-fef0-4794-8d5f-ffd886f610bf/insights-operator/1.log" Jul 06 15:04:51.105784 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:51.105761 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xkc2v_7bb9e974-a960-49aa-afac-23abcf7b6165/kube-rbac-proxy/0.log" Jul 06 15:04:51.132695 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:51.132672 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xkc2v_7bb9e974-a960-49aa-afac-23abcf7b6165/exporter/0.log" Jul 06 15:04:51.156738 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:51.156664 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xkc2v_7bb9e974-a960-49aa-afac-23abcf7b6165/extractor/0.log" Jul 06 15:04:51.682778 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:51.682751 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gbbmr/perf-node-gather-daemonset-7nzkf" Jul 06 15:04:53.920188 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:53.920157 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ff654ddb6-74kvm_79276d4e-dd7e-4e87-b7f9-764025f69455/maas-api/0.log" Jul 06 15:04:54.068507 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:54.068476 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74787c7b7f-rd2ht_5c7036fb-2724-407b-80cb-69816c1a17cc/manager/0.log" Jul 06 15:04:54.094633 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:54.094606 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xk4x2_666c79f7-e3be-4bbc-9cd5-13fe98bb2985/server/0.log" Jul 06 15:04:54.165860 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:54.165812 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gnznn_e8dfbec5-bf40-4b90-b72b-942679b69c68/manager/0.log" Jul 06 15:04:54.323826 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:54.323747 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-557ddcc9b5-vttbb_36bf9105-a209-4b24-b346-85fe501c784e/manager/0.log" Jul 06 15:04:54.351422 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:54.351394 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-557ddcc9b5-vttbb_36bf9105-a209-4b24-b346-85fe501c784e/copy-manifests/0.log" Jul 06 15:04:55.330753 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:55.330723 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b5f845fbc-bvglr_cb74fabc-8555-437c-8959-7f50d3b3553e/kube-auth-proxy/0.log" Jul 06 15:04:55.406859 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:55.406824 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9fkt2_f2560319-f9b2-49e8-8a42-13c6308d9fd2/payload-pre-processing/0.log" Jul 06 15:04:55.799539 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:55.799509 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66c4cbf4f4-glf8z_ffedf392-55a8-4d69-9a47-d6afdedcb8c8/manager/0.log" Jul 06 15:04:56.880670 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:56.880629 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b5f845fbc-bvglr_cb74fabc-8555-437c-8959-7f50d3b3553e/kube-auth-proxy/0.log" Jul 06 15:04:56.966014 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:04:56.965984 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9fkt2_f2560319-f9b2-49e8-8a42-13c6308d9fd2/payload-pre-processing/0.log" Jul 06 15:05:00.545196 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:05:00.545158 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2wztw_8190d330-b0cf-4771-8894-a35c0e43fbb6/migrator/0.log" Jul 06 15:05:00.565525 ip-10-0-141-213 kubenswrapper[2552]: I0706 15:05:00.565499 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2wztw_8190d330-b0cf-4771-8894-a35c0e43fbb6/graceful-termination/0.log"