Jul 13 16:39:45.080336 ip-10-0-141-178 systemd[1]: Starting Kubernetes Kubelet... Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.539505 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543321 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543332 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543336 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543340 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543374 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543378 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543382 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:45.549096 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543385 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543389 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543391 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543394 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543396 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543400 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543404 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543407 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543410 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543412 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543415 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543418 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543421 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543424 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543427 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543430 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543432 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543435 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543437 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:45.564720 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543440 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543444 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543446 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543449 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543451 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543454 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543457 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543459 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543462 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543466 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543469 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543472 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543475 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543479 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543482 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543485 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543488 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543491 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543494 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:45.566336 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543498 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543500 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543503 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543506 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543508 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543511 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543513 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543516 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543518 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543521 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543524 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543527 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543529 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543532 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543534 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543537 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543539 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543542 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543544 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543546 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:45.586068 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543549 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543552 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543554 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543556 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543559 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543561 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543564 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543567 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543569 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543571 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543574 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543576 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543579 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543582 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543585 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543587 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543590 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543592 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543595 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543597 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:45.589574 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543600 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543602 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543605 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543607 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543609 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543612 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543614 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543617 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543619 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543622 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543624 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543626 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543629 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.543631 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544323 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544329 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544332 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544335 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544338 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544341 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:45.599714 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544344 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544349 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544353 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544355 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544359 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544362 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544365 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544367 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544370 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544374 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544377 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544381 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544384 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544387 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544390 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544393 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544396 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544399 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544402 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:45.602903 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544404 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544407 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544409 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544412 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544414 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544417 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544420 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544422 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544425 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544427 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544430 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544432 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544435 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544443 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544446 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544449 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544451 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544453 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544456 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544458 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:45.613153 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544461 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544464 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544466 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544470 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544473 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544476 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544479 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544481 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544484 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544486 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544489 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544492 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544494 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544496 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544499 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544501 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544505 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544508 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544510 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544513 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:45.617084 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544515 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544518 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544520 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544523 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544525 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544528 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544531 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544534 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544536 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544539 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544541 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544544 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544546 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544549 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544552 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544554 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544557 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544559 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544562 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544564 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:45.638486 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544566 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:45.618176 ip-10-0-141-178 systemd[1]: Started Kubernetes Kubelet. Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544569 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544572 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544574 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544577 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544579 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544581 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544584 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544587 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544589 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544592 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544594 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544597 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544599 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544969 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544974 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544978 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544981 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544984 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544987 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:45.640799 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544990 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544992 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544995 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.544998 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545000 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545003 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545007 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545010 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545013 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545016 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545019 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545022 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545025 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545027 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545031 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545051 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545055 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545058 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545063 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:45.816970 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545066 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545069 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545072 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545074 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545077 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545080 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545083 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545085 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545088 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545091 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545094 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545096 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545099 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545102 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545105 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545107 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545110 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545112 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545117 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545119 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:45.831245 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545122 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545124 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545126 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545129 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545132 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545134 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545137 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545139 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545142 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545144 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545146 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545152 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545155 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545157 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545160 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545163 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545165 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545168 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545170 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545173 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:45.832443 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545175 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545178 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545180 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545183 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545185 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545188 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545191 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545193 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545195 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545198 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545202 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545205 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545219 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545222 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545225 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545228 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545231 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545233 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545236 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545238 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:45.838054 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545242 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545244 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545247 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545251 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545253 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545256 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545260 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545263 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545265 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545268 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545270 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545273 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545275 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545278 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545344 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545352 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545361 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545369 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545375 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545380 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 16:39:45.838940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545385 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545392 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545398 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545401 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545404 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545407 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545410 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545413 2569 flags.go:64] FLAG: --cgroup-root="" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545416 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545419 2569 flags.go:64] FLAG: --client-ca-file="" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545421 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545425 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545428 2569 flags.go:64] FLAG: --cluster-domain="" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545431 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545434 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545439 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545442 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545446 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545449 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545452 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545455 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545458 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545461 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545464 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 16:39:45.840912 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545467 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545470 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545473 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545476 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545479 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545482 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545485 2569 flags.go:64] FLAG: --enable-server="true" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545489 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545494 2569 flags.go:64] FLAG: --event-burst="100" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545496 2569 flags.go:64] FLAG: --event-qps="50" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545501 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545504 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545507 2569 flags.go:64] FLAG: --eviction-hard="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545510 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545513 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545516 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545519 2569 flags.go:64] FLAG: --eviction-soft="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545522 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545525 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545528 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545531 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545534 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545536 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545540 2569 flags.go:64] FLAG: --feature-gates="" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545544 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 16:39:45.845702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545547 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545550 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545553 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545556 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545559 2569 flags.go:64] FLAG: --help="false" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545561 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-141-178.ec2.internal" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545565 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545567 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545570 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545574 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545577 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545581 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545583 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545586 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545589 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545592 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545596 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545600 2569 flags.go:64] FLAG: --kube-reserved="" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545603 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545606 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545609 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545611 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545614 2569 flags.go:64] FLAG: --lock-file="" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545617 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 16:39:45.848384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545620 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545622 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545627 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545630 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545633 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545636 2569 flags.go:64] FLAG: --logging-format="text" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545639 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545643 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545645 2569 flags.go:64] FLAG: --manifest-url="" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545648 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545652 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545655 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545659 2569 flags.go:64] FLAG: --max-pods="110" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545662 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545665 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545667 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545670 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545673 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545676 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545679 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545686 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545690 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545693 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 16:39:45.851942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545695 2569 flags.go:64] FLAG: --pod-cidr="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545699 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545707 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545710 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545713 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545716 2569 flags.go:64] FLAG: --port="10250" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545719 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545721 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb6634fced8fd81c" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545724 2569 flags.go:64] FLAG: --qos-reserved="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545727 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545730 2569 flags.go:64] FLAG: --register-node="true" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545733 2569 flags.go:64] FLAG: --register-with-taints="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545736 2569 flags.go:64] FLAG: --registry-burst="10" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545739 2569 flags.go:64] FLAG: --registry-qps="5" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545742 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545745 2569 flags.go:64] FLAG: --reserved-memory="" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545751 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545754 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545757 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545760 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545763 2569 flags.go:64] FLAG: --runonce="false" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545765 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545768 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545772 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545775 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 16:39:45.852775 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545778 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545781 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545784 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545787 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545790 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545792 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545795 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545798 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545801 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545806 2569 flags.go:64] FLAG: --system-cgroups="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545809 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545814 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545817 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545820 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545823 2569 flags.go:64] FLAG: --tls-min-version="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545826 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545829 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545832 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545835 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545838 2569 flags.go:64] FLAG: --v="2" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545842 2569 flags.go:64] FLAG: --version="false" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545846 2569 flags.go:64] FLAG: --vmodule="" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545849 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.545854 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545974 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:45.855024 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545978 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545981 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545985 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545987 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545991 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545994 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545996 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.545999 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546001 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546003 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546006 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546009 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546011 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546014 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546017 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546019 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546023 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546026 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546029 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546032 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:45.855757 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546049 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546052 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546055 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546058 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546060 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546063 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546065 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546068 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546071 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546073 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546077 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546080 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546082 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546086 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546090 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546093 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546095 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546098 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546101 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:45.996013 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546103 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546106 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546108 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546111 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546113 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546116 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546118 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546121 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546123 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546127 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546129 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546133 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546135 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546138 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546140 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546143 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546145 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546148 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546150 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546153 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:45.996659 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546155 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546158 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546160 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546165 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546167 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546170 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546172 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546175 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546177 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546180 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546184 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546187 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546190 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546193 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546195 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546198 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546200 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546203 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546205 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:45.997251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546208 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546210 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546214 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546216 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546219 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546229 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546231 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546234 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546236 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546239 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546241 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546245 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546248 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546251 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546254 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546257 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546261 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546263 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546266 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:45.997852 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.546268 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.547136 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.555303 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.555323 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.555346 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.555354 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555421 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555426 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555433 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555436 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555439 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555442 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555449 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555453 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:45.998394 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555455 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555458 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555461 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555464 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555466 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555473 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555476 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555478 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555481 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555484 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555487 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555489 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555495 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555498 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555501 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555503 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555507 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555510 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555517 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:45.998840 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555520 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555522 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555530 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555533 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555539 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555542 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555545 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555547 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555550 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555552 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555559 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555562 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555564 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555567 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555571 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555579 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555582 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555585 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555588 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:45.999396 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555591 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555598 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555601 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555604 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555607 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555609 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555612 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555615 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555621 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555624 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555626 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555629 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555632 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555638 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555641 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555645 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555649 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555651 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555654 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:45.999931 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555662 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555665 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555669 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555672 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555674 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555677 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555684 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555686 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555689 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555691 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555694 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555696 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555699 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555705 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555708 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555710 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555713 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555715 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555718 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555721 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:46.000598 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555725 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555728 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555734 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555737 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555740 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555743 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555745 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555748 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555752 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555758 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555762 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555764 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555767 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555770 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.555775 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 16:39:46.001145 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555921 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555929 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555932 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555935 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555940 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555943 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555946 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555948 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555955 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555957 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555960 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555965 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555968 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555970 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555978 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555982 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555988 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555991 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555994 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 16:39:46.001521 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.555999 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556002 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556014 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556017 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556019 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556024 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556027 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556031 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556034 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556050 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556053 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556055 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556058 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556065 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556068 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556070 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556073 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556075 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556078 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 16:39:46.002004 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556084 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556087 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556090 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556092 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556095 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556097 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556104 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556107 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556109 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556112 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556115 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556117 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556124 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556126 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556129 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556133 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556135 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556138 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556140 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556147 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 16:39:46.002469 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556152 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556155 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556157 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556160 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556163 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556170 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556172 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556175 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556177 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556180 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556182 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556189 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556191 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556194 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556197 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556199 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556202 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556204 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556210 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556213 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556215 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 16:39:46.002965 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556218 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556221 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556223 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556230 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556234 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556237 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556241 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556244 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556250 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556253 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556255 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556261 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556264 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556271 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556274 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556277 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556280 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556283 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556289 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 16:39:46.003502 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:45.556292 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.556297 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.556415 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.558999 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.560718 2569 server.go:1019] "Starting client certificate rotation" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.560831 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.560869 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.564276 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.573206 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.583256 2569 log.go:25] "Validated CRI v1 runtime API" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.585464 2569 log.go:25] "Validated CRI v1 image API" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.586348 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 16:39:46.003951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.588721 2569 fs.go:135] Filesystem UUIDs: map[0b97d547-7547-479d-8b3b-01424805c9d4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ae2ef758-c635-4af3-ba54-278523cd97a3:/dev/nvme0n1p4] Jul 13 16:39:46.004269 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.588741 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.592819 2569 manager.go:216] Machine: {Timestamp:2026-07-13 16:39:45.592327129 +0000 UTC m=+0.382415054 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101270 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f9bdec46e2c4711e31cac66e8371e SystemUUID:ec2f9bde-c46e-2c47-11e3-1cac66e8371e BootID:26af1651-bede-4b4c-ba9f-250d23844478 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8b:df:3d:63:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8b:df:3d:63:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:9a:d0:4c:2e:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.592912 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.593011 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.596481 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.596516 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.597462 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.597473 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 16:39:46.004299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.597480 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.598178 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.599854 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.599856 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.603879 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604019 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604062 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604074 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604085 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604096 2569 kubelet.go:398] "Adding apiserver pod source" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604106 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604442 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.604465 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.606852 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609161 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609191 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609341 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609355 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609361 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609369 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609377 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609383 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609979 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609988 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.609997 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.610003 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.610013 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.610022 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.611086 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.611097 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.617353 2569 server.go:1268] "Started kubelet" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.617673 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.618090 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.618182 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.618854 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.622957 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.624312 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.624364 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.624553 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 16:39:46.004529 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.625131 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.633372 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.633474 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.634140 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634871 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634885 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634874 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634921 2569 factory.go:55] Registering systemd factory Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634939 2569 factory.go:223] Registration of the systemd container factory successfully Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634980 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.634993 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.636180 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.636328 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.636184 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-178.ec2.internal.18c1e6e1f7f297c5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-178.ec2.internal,UID:ip-10-0-141-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-178.ec2.internal,},FirstTimestamp:2026-07-13 16:39:45.617328069 +0000 UTC m=+0.407415994,LastTimestamp:2026-07-13 16:39:45.617328069 +0000 UTC m=+0.407415994,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-178.ec2.internal,}" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.637867 2569 factory.go:153] Registering CRI-O factory Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.637882 2569 factory.go:223] Registration of the crio container factory successfully Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.638122 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.638156 2569 factory.go:103] Registering Raw factory Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.638169 2569 manager.go:1196] Started watching for new ooms in manager Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.638492 2569 manager.go:349] Starting recovery of all containers Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.648285 2569 manager.go:354] Recovery completed Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.649595 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdb55" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.652331 2569 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.655445 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.655801 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vdb55" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657388 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657421 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657431 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657926 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657941 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657956 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.657992 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.005516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.658693 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.658719 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.658734 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.658985 2569 policy_none.go:49] "None policy: Start" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.659001 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.659013 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.660125 2569 policy_none.go:47] "Start" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.687613 2569 manager.go:337] "Starting Device Plugin manager" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.687816 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.687830 2569 server.go:87] "Starting device plugin registration server" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.688269 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.688283 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.688381 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.688457 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.688465 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.688990 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.689049 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.712416 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.713621 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.713657 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.713677 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.713707 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.715318 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.789553 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.790452 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.790482 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.790497 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.790523 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.800919 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.800937 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-178.ec2.internal\": node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.814670 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal"] Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.814721 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.819221 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.819241 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.819250 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.820513 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.820631 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.820662 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.006354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821248 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821268 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821288 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821315 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821270 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.821361 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.822541 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.822565 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.823203 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientMemory" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.823222 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.823234 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeHasSufficientPID" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.855976 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.877638 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-178.ec2.internal\" not found" node="ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.883058 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-178.ec2.internal\" not found" node="ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:45.956388 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.956429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.956450 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.007378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:45.956469 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c9b46fdf787e2321bc5c8391e702436-config\") pod \"kube-apiserver-proxy-ip-10-0-141-178.ec2.internal\" (UID: \"1c9b46fdf787e2321bc5c8391e702436\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057394 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.057364 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.057466 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057466 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057466 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057455 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c9b46fdf787e2321bc5c8391e702436-config\") pod \"kube-apiserver-proxy-ip-10-0-141-178.ec2.internal\" (UID: \"1c9b46fdf787e2321bc5c8391e702436\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057549 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057498 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c9b46fdf787e2321bc5c8391e702436-config\") pod \"kube-apiserver-proxy-ip-10-0-141-178.ec2.internal\" (UID: \"1c9b46fdf787e2321bc5c8391e702436\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057549 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057516 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.057549 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.057521 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f270f0b2772721148249b750c246fb3f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal\" (UID: \"f270f0b2772721148249b750c246fb3f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.158141 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.158065 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.180248 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.180227 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.187586 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.187566 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.258980 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.258940 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.359281 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.359243 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.459702 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.459637 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-178.ec2.internal\" not found" Jul 13 16:39:46.466749 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.466734 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 16:39:46.535254 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.535230 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.552616 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.552600 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 16:39:46.554914 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.554899 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.560315 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.560298 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 16:39:46.560432 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.560413 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 16:39:46.560522 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.560484 2569 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://ad5f7fbb8ecec4dc49b4ee3dbc1939fd-525a7197dc6350ff.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.141.178:33568->100.58.19.130:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" Jul 13 16:39:46.606106 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.606073 2569 apiserver.go:52] "Watching apiserver" Jul 13 16:39:46.617120 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.617098 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 16:39:46.619347 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.619326 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-qjvcv","openshift-ovn-kubernetes/ovnkube-node-v7jxg","kube-system/konnectivity-agent-nbzzc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx","openshift-dns/node-resolver-m2ppc","openshift-image-registry/node-ca-7m9tz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal","openshift-multus/multus-2jlz5","openshift-multus/network-metrics-daemon-4r5xc","openshift-network-diagnostics/network-check-target-254w7","kube-system/global-pull-secret-syncer-grd9b","kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5zv66","openshift-multus/multus-additional-cni-plugins-rw5cq"] Jul 13 16:39:46.620812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.620795 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.622080 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.622060 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.623299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.623280 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.623884 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.623863 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.623993 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.623977 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fl5bb\"" Jul 13 16:39:46.624527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.624512 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.624527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.624523 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.624754 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.624732 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 16:39:46.624837 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.624760 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 16:39:46.625194 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.625180 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 16:39:46.625272 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.625258 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.625783 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.625767 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.626869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.626847 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 16:39:46.626974 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.626954 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 16:39:46.628148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.627256 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 16:39:46.628148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.627432 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.628148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.627474 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mr84s\"" Jul 13 16:39:46.628148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.627725 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.628148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.627756 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p5hp7\"" Jul 13 16:39:46.628411 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.628330 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 16:39:46.629729 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.629704 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.630695 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630671 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.630863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630707 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.630863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630712 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.630863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 16:39:46.630863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630727 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.631115 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630905 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5qcxg\"" Jul 13 16:39:46.631115 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.630978 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 16:39:46.631115 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.631102 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.631273 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.631138 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qgh8c\"" Jul 13 16:39:46.631273 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.631168 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:46.631273 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.631191 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.631412 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.631296 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pfxhh\"" Jul 13 16:39:46.631454 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.631423 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.633022 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.633007 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:46.633129 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.633075 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:46.634341 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.634323 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 16:39:46.634423 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.634344 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.634488 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.634423 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:46.634962 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.634942 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.635047 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.634985 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 16:39:46.635047 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.634944 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 16:39:46.635169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.635138 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.635230 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.635215 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9rw5l\"" Jul 13 16:39:46.635715 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.635701 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.636942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.636928 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.642444 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.642426 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 16:39:46.642523 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.642510 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 16:39:46.642668 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.642657 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-trf2l\"" Jul 13 16:39:46.642668 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.642664 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:39:46.643312 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.643295 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9zdps\"" Jul 13 16:39:46.643397 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.643352 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 16:39:46.659129 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.659068 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 16:34:46 +0000 UTC" deadline="2028-04-25 20:21:52.75975988 +0000 UTC" Jul 13 16:39:46.659129 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.659117 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15651h42m6.100645161s" Jul 13 16:39:46.660829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660816 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-netns\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.660892 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-hostroot\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.660892 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660857 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.660892 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660881 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8647d58d-2479-44a1-892d-4ee82ebcf4ef-iptables-alerter-script\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.660985 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc25d\" (UniqueName: \"kubernetes.io/projected/10036d42-4c47-4162-afb3-a04d6c213a01-kube-api-access-kc25d\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.660985 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7p4\" (UniqueName: \"kubernetes.io/projected/c5b7a469-647c-4afe-91a5-e11edfb96827-kube-api-access-2h7p4\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.660985 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660944 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-socket-dir-parent\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.660985 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.660966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-bin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661113 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661010 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.661113 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-netd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661113 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661068 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-konnectivity-ca\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.661113 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661089 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysconfig\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.661113 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-multus-daemon-config\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661121 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-systemd-units\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661136 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-etc-tuned\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661163 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znjv2\" (UniqueName: \"kubernetes.io/projected/f6b20c96-8862-45d6-882e-d20a22dba057-kube-api-access-znjv2\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661178 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661193 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-registration-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661232 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-cnibin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-kubelet\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661266 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-var-lib-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-sys-fs\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661300 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-cni-binary-copy\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661332 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-multus\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-systemd\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661388 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b7a469-647c-4afe-91a5-e11edfb96827-host\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-kubelet\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661421 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-agent-certs\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661435 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744ln\" (UniqueName: \"kubernetes.io/projected/4cc7aca3-ab5d-4c42-af22-45450b82be31-kube-api-access-744ln\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.661483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661459 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-var-lib-kubelet\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661508 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z88r\" (UniqueName: \"kubernetes.io/projected/8647d58d-2479-44a1-892d-4ee82ebcf4ef-kube-api-access-9z88r\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661530 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-os-release\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661546 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-conf-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661563 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-device-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661586 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-config\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661608 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-dbus\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661633 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-k8s-cni-cncf-io\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661655 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-etc-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-bin\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661714 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-script-lib\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661741 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrcz\" (UniqueName: \"kubernetes.io/projected/5d19c188-a7a4-415e-b9ae-2446e3399c85-kube-api-access-hjrcz\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661763 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-kubelet-config\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.661780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661777 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-conf\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-multus-certs\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661814 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ldl4\" (UniqueName: \"kubernetes.io/projected/163640c8-2bec-4876-a451-71fd060c9c9f-kube-api-access-6ldl4\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661844 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661879 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-log-socket\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661899 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-sys\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-host\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661964 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8647d58d-2479-44a1-892d-4ee82ebcf4ef-host-slash\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.661993 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662014 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcx7l\" (UniqueName: \"kubernetes.io/projected/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-kube-api-access-dcx7l\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662038 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-systemd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662074 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-env-overrides\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662108 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7aca3-ab5d-4c42-af22-45450b82be31-tmp-dir\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662136 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-modprobe-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662162 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-tmp\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662220 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-netns\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662277 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-ovn\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662323 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-node-log\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662337 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-run\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662368 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-lib-modules\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662396 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-socket-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5b7a469-647c-4afe-91a5-e11edfb96827-serviceca\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662444 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662471 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662496 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-kubernetes\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662521 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.662869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662554 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-system-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.663283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662581 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovn-node-metrics-cert\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.663283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662605 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-etc-kubernetes\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.663283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662630 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-slash\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.663283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.662667 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7aca3-ab5d-4c42-af22-45450b82be31-hosts-file\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.663283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.663238 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 16:39:46.717272 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.717251 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cgjxw" Jul 13 16:39:46.729384 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.729365 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cgjxw" Jul 13 16:39:46.735418 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.735391 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 16:39:46.763320 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763429 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763359 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.763429 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-system-cni-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.763429 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763407 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cnibin\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763428 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-registration-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763452 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-cnibin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763476 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-kubelet\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763502 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-var-lib-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-sys-fs\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763547 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-kubelet\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763582 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763567 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763574 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-registration-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763604 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-var-lib-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763547 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763570 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-cnibin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-cni-binary-copy\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763665 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-multus\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763633 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-sys-fs\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763688 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-systemd\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763738 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-multus\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763779 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b7a469-647c-4afe-91a5-e11edfb96827-host\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763808 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-kubelet\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.763829 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763819 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-systemd\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763834 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-agent-certs\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763851 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5b7a469-647c-4afe-91a5-e11edfb96827-host\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763861 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744ln\" (UniqueName: \"kubernetes.io/projected/4cc7aca3-ab5d-4c42-af22-45450b82be31-kube-api-access-744ln\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763886 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-var-lib-kubelet\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763888 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-kubelet\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763910 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z88r\" (UniqueName: \"kubernetes.io/projected/8647d58d-2479-44a1-892d-4ee82ebcf4ef-kube-api-access-9z88r\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.763947 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-var-lib-kubelet\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-os-release\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-conf-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764155 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-device-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764169 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-cni-binary-copy\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764180 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-config\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764197 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-conf-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.764382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764195 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764203 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-dbus\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764263 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-device-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764268 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-os-release\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764269 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764314 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-dbus\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764322 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg4gl\" (UniqueName: \"kubernetes.io/projected/72abcb4e-bd9b-4846-85c4-132f6a46fabf-kube-api-access-dg4gl\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764361 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-k8s-cni-cncf-io\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-etc-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764434 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-bin\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764447 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-k8s-cni-cncf-io\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764470 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-etc-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-script-lib\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764521 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrcz\" (UniqueName: \"kubernetes.io/projected/5d19c188-a7a4-415e-b9ae-2446e3399c85-kube-api-access-hjrcz\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764526 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-bin\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764561 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-kubelet-config\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-conf\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-multus-certs\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764643 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ldl4\" (UniqueName: \"kubernetes.io/projected/163640c8-2bec-4876-a451-71fd060c9c9f-kube-api-access-6ldl4\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764668 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764691 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81257490-18b6-48fa-8601-eef4639687e1-kubelet-config\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764693 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-log-socket\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764699 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-multus-certs\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-sys\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764744 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-openvswitch\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764748 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-config\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764747 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-host\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764726 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-log-socket\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764778 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-sys\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.765883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764785 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8647d58d-2479-44a1-892d-4ee82ebcf4ef-host-slash\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764793 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-conf\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764815 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8647d58d-2479-44a1-892d-4ee82ebcf4ef-host-slash\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764815 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764862 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcx7l\" (UniqueName: \"kubernetes.io/projected/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-kube-api-access-dcx7l\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764895 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-systemd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764902 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-host\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764931 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-env-overrides\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764952 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7aca3-ab5d-4c42-af22-45450b82be31-tmp-dir\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764973 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-etc-selinux\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.764995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-modprobe-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-tmp\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765047 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-os-release\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.766812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765075 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765101 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-netns\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765133 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-systemd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765146 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-ovn\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765172 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-node-log\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-run\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765221 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-lib-modules\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765249 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-socket-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765256 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc7aca3-ab5d-4c42-af22-45450b82be31-tmp-dir\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5b7a469-647c-4afe-91a5-e11edfb96827-serviceca\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765303 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765330 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765346 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-run\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765373 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-env-overrides\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765385 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-kubernetes\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765388 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-run-netns\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765411 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765434 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-run-ovn\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765496 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-lib-modules\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765432 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-modprobe-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765435 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-system-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.765527 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765545 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-socket-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovn-node-metrics-cert\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765616 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.765646 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:47.265577198 +0000 UTC m=+2.055665269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765668 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-etc-kubernetes\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765682 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10036d42-4c47-4162-afb3-a04d6c213a01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-slash\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.767795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765707 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovnkube-script-lib\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765726 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7aca3-ab5d-4c42-af22-45450b82be31-hosts-file\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765737 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-kubernetes\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-netns\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765779 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-node-log\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765784 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-hostroot\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765837 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5b7a469-647c-4afe-91a5-e11edfb96827-serviceca\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765837 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-slash\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765871 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-etc-kubernetes\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765908 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-run-netns\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765938 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysctl-d\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765937 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-system-cni-dir\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765958 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc7aca3-ab5d-4c42-af22-45450b82be31-hosts-file\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8647d58d-2479-44a1-892d-4ee82ebcf4ef-iptables-alerter-script\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.768441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.765980 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-hostroot\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766003 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc25d\" (UniqueName: \"kubernetes.io/projected/10036d42-4c47-4162-afb3-a04d6c213a01-kube-api-access-kc25d\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7p4\" (UniqueName: \"kubernetes.io/projected/c5b7a469-647c-4afe-91a5-e11edfb96827-kube-api-access-2h7p4\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766061 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-socket-dir-parent\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-bin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766131 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766190 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-multus-socket-dir-parent\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.766223 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.766288 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:47.266269224 +0000 UTC m=+2.056357146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766337 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/163640c8-2bec-4876-a451-71fd060c9c9f-host-var-lib-cni-bin\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766355 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-netd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766400 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-host-cni-netd\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766401 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-konnectivity-ca\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysconfig\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.768861 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766472 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-multus-daemon-config\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766485 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8647d58d-2479-44a1-892d-4ee82ebcf4ef-iptables-alerter-script\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766497 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-systemd-units\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766522 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-etc-tuned\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766550 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6b20c96-8862-45d6-882e-d20a22dba057-etc-sysconfig\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766564 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znjv2\" (UniqueName: \"kubernetes.io/projected/f6b20c96-8862-45d6-882e-d20a22dba057-kube-api-access-znjv2\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766607 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d19c188-a7a4-415e-b9ae-2446e3399c85-systemd-units\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.766638 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.767000 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-konnectivity-ca\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.767019 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/163640c8-2bec-4876-a451-71fd060c9c9f-multus-daemon-config\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.768091 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-tmp\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.768205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d19c188-a7a4-415e-b9ae-2446e3399c85-ovn-node-metrics-cert\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.768341 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f-agent-certs\") pod \"konnectivity-agent-nbzzc\" (UID: \"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f\") " pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.769276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.768737 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6b20c96-8862-45d6-882e-d20a22dba057-etc-tuned\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.775831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.775799 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ldl4\" (UniqueName: \"kubernetes.io/projected/163640c8-2bec-4876-a451-71fd060c9c9f-kube-api-access-6ldl4\") pod \"multus-2jlz5\" (UID: \"163640c8-2bec-4876-a451-71fd060c9c9f\") " pod="openshift-multus/multus-2jlz5" Jul 13 16:39:46.775901 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.775835 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcx7l\" (UniqueName: \"kubernetes.io/projected/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-kube-api-access-dcx7l\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:46.778640 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.778614 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrcz\" (UniqueName: \"kubernetes.io/projected/5d19c188-a7a4-415e-b9ae-2446e3399c85-kube-api-access-hjrcz\") pod \"ovnkube-node-v7jxg\" (UID: \"5d19c188-a7a4-415e-b9ae-2446e3399c85\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.779989 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.779968 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-744ln\" (UniqueName: \"kubernetes.io/projected/4cc7aca3-ab5d-4c42-af22-45450b82be31-kube-api-access-744ln\") pod \"node-resolver-m2ppc\" (UID: \"4cc7aca3-ab5d-4c42-af22-45450b82be31\") " pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:46.780503 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.780480 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7p4\" (UniqueName: \"kubernetes.io/projected/c5b7a469-647c-4afe-91a5-e11edfb96827-kube-api-access-2h7p4\") pod \"node-ca-7m9tz\" (UID: \"c5b7a469-647c-4afe-91a5-e11edfb96827\") " pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:46.791070 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.791036 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:39:46.791166 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.791076 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:39:46.791166 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.791092 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:46.791297 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:46.791176 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:39:47.291158612 +0000 UTC m=+2.081246545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:46.793251 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.793232 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znjv2\" (UniqueName: \"kubernetes.io/projected/f6b20c96-8862-45d6-882e-d20a22dba057-kube-api-access-znjv2\") pod \"tuned-5zv66\" (UID: \"f6b20c96-8862-45d6-882e-d20a22dba057\") " pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:46.793325 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.793290 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc25d\" (UniqueName: \"kubernetes.io/projected/10036d42-4c47-4162-afb3-a04d6c213a01-kube-api-access-kc25d\") pod \"aws-ebs-csi-driver-node-sfzdx\" (UID: \"10036d42-4c47-4162-afb3-a04d6c213a01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:46.805706 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.805682 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z88r\" (UniqueName: \"kubernetes.io/projected/8647d58d-2479-44a1-892d-4ee82ebcf4ef-kube-api-access-9z88r\") pod \"iptables-alerter-qjvcv\" (UID: \"8647d58d-2479-44a1-892d-4ee82ebcf4ef\") " pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.867495 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867472 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867589 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867500 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-os-release\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867589 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867694 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-system-cni-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867694 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867618 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-os-release\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867694 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867627 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cnibin\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867694 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867670 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cnibin\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867694 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867682 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867716 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-system-cni-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867729 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg4gl\" (UniqueName: \"kubernetes.io/projected/72abcb4e-bd9b-4846-85c4-132f6a46fabf-kube-api-access-dg4gl\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.867949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.867761 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72abcb4e-bd9b-4846-85c4-132f6a46fabf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.868224 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.868205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.868786 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.868766 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-cni-binary-copy\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.868875 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.868860 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72abcb4e-bd9b-4846-85c4-132f6a46fabf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.880579 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.880560 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg4gl\" (UniqueName: \"kubernetes.io/projected/72abcb4e-bd9b-4846-85c4-132f6a46fabf-kube-api-access-dg4gl\") pod \"multus-additional-cni-plugins-rw5cq\" (UID: \"72abcb4e-bd9b-4846-85c4-132f6a46fabf\") " pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:46.949214 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.949190 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qjvcv" Jul 13 16:39:46.966567 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.966504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:39:46.979358 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.979339 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:39:46.985264 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:46.985245 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" Jul 13 16:39:47.013763 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.013733 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2ppc" Jul 13 16:39:47.014210 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.014189 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 16:39:47.027653 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.027632 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7m9tz" Jul 13 16:39:47.038264 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.038239 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2jlz5" Jul 13 16:39:47.044050 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.044023 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5zv66" Jul 13 16:39:47.057822 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.057798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" Jul 13 16:39:47.138582 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.138549 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf270f0b2772721148249b750c246fb3f.slice/crio-df92df79a411fb7398e54a88a267488f33794ba631b7fc1bbc17280ccb093e98 WatchSource:0}: Error finding container df92df79a411fb7398e54a88a267488f33794ba631b7fc1bbc17280ccb093e98: Status 404 returned error can't find the container with id df92df79a411fb7398e54a88a267488f33794ba631b7fc1bbc17280ccb093e98 Jul 13 16:39:47.139224 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.139201 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9b46fdf787e2321bc5c8391e702436.slice/crio-4499037949fd2d0b5d86d71636140beeae83544038593cf6f6d6fd33d1ab692a WatchSource:0}: Error finding container 4499037949fd2d0b5d86d71636140beeae83544038593cf6f6d6fd33d1ab692a: Status 404 returned error can't find the container with id 4499037949fd2d0b5d86d71636140beeae83544038593cf6f6d6fd33d1ab692a Jul 13 16:39:47.144007 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.143992 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 16:39:47.200348 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.200328 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 16:39:47.273423 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.273344 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:47.273423 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.273418 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:47.273615 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.273510 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:47.273615 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.273510 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:47.273615 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.273580 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:48.273558411 +0000 UTC m=+3.063646348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:47.273787 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.273625 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:48.273608213 +0000 UTC m=+3.063696127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:47.373946 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.373908 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:47.374142 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.374121 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:39:47.374221 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.374154 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:39:47.374221 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.374168 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:47.374333 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.374235 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:39:48.37421478 +0000 UTC m=+3.164302709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:47.377353 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.377332 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163640c8_2bec_4876_a451_71fd060c9c9f.slice/crio-9dadc0d9896a8ab70cc1e6abbbd3e0b72d4ea7e9ebd018ef185cc4a2adf25597 WatchSource:0}: Error finding container 9dadc0d9896a8ab70cc1e6abbbd3e0b72d4ea7e9ebd018ef185cc4a2adf25597: Status 404 returned error can't find the container with id 9dadc0d9896a8ab70cc1e6abbbd3e0b72d4ea7e9ebd018ef185cc4a2adf25597 Jul 13 16:39:47.414533 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.414509 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e58ff0_43a6_464f_a81a_7a1a3d8a8b9f.slice/crio-87b83355ee90c3205c395273e1e2b9ce14894d0a69caf44794e59284cdc33503 WatchSource:0}: Error finding container 87b83355ee90c3205c395273e1e2b9ce14894d0a69caf44794e59284cdc33503: Status 404 returned error can't find the container with id 87b83355ee90c3205c395273e1e2b9ce14894d0a69caf44794e59284cdc33503 Jul 13 16:39:47.464826 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.464803 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b7a469_647c_4afe_91a5_e11edfb96827.slice/crio-c82050a6c56a07d1fc94bb9b644d495257352e7eb5c5b4ab89734cbcc4c1cd74 WatchSource:0}: Error finding container c82050a6c56a07d1fc94bb9b644d495257352e7eb5c5b4ab89734cbcc4c1cd74: Status 404 returned error can't find the container with id c82050a6c56a07d1fc94bb9b644d495257352e7eb5c5b4ab89734cbcc4c1cd74 Jul 13 16:39:47.477169 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.477144 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc7aca3_ab5d_4c42_af22_45450b82be31.slice/crio-e217438fd9269900b44a404a5cc201e97534f46fd94b7b3f560632136ddcf6fc WatchSource:0}: Error finding container e217438fd9269900b44a404a5cc201e97534f46fd94b7b3f560632136ddcf6fc: Status 404 returned error can't find the container with id e217438fd9269900b44a404a5cc201e97534f46fd94b7b3f560632136ddcf6fc Jul 13 16:39:47.483517 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.483496 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72abcb4e_bd9b_4846_85c4_132f6a46fabf.slice/crio-49f4a28ac273ee9fbb923010836c0260b47e60d617bc421e5c5921ad6720c264 WatchSource:0}: Error finding container 49f4a28ac273ee9fbb923010836c0260b47e60d617bc421e5c5921ad6720c264: Status 404 returned error can't find the container with id 49f4a28ac273ee9fbb923010836c0260b47e60d617bc421e5c5921ad6720c264 Jul 13 16:39:47.490304 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.490284 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10036d42_4c47_4162_afb3_a04d6c213a01.slice/crio-a56b5ca34042a6b90fdb6e2691eefde6e8516000cb4faa89a53a6e968a040917 WatchSource:0}: Error finding container a56b5ca34042a6b90fdb6e2691eefde6e8516000cb4faa89a53a6e968a040917: Status 404 returned error can't find the container with id a56b5ca34042a6b90fdb6e2691eefde6e8516000cb4faa89a53a6e968a040917 Jul 13 16:39:47.581760 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.581689 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 16:39:47.714388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.714362 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:47.714512 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:47.714475 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:47.718093 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.718030 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerStarted","Data":"49f4a28ac273ee9fbb923010836c0260b47e60d617bc421e5c5921ad6720c264"} Jul 13 16:39:47.719094 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.719065 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2ppc" event={"ID":"4cc7aca3-ab5d-4c42-af22-45450b82be31","Type":"ContainerStarted","Data":"e217438fd9269900b44a404a5cc201e97534f46fd94b7b3f560632136ddcf6fc"} Jul 13 16:39:47.720027 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.719999 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7m9tz" event={"ID":"c5b7a469-647c-4afe-91a5-e11edfb96827","Type":"ContainerStarted","Data":"c82050a6c56a07d1fc94bb9b644d495257352e7eb5c5b4ab89734cbcc4c1cd74"} Jul 13 16:39:47.721067 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.721041 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbzzc" event={"ID":"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f","Type":"ContainerStarted","Data":"87b83355ee90c3205c395273e1e2b9ce14894d0a69caf44794e59284cdc33503"} Jul 13 16:39:47.722107 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.722090 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2jlz5" event={"ID":"163640c8-2bec-4876-a451-71fd060c9c9f","Type":"ContainerStarted","Data":"9dadc0d9896a8ab70cc1e6abbbd3e0b72d4ea7e9ebd018ef185cc4a2adf25597"} Jul 13 16:39:47.723030 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.723010 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" event={"ID":"1c9b46fdf787e2321bc5c8391e702436","Type":"ContainerStarted","Data":"4499037949fd2d0b5d86d71636140beeae83544038593cf6f6d6fd33d1ab692a"} Jul 13 16:39:47.724059 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.724026 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" event={"ID":"f270f0b2772721148249b750c246fb3f","Type":"ContainerStarted","Data":"df92df79a411fb7398e54a88a267488f33794ba631b7fc1bbc17280ccb093e98"} Jul 13 16:39:47.725036 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.725017 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" event={"ID":"10036d42-4c47-4162-afb3-a04d6c213a01","Type":"ContainerStarted","Data":"a56b5ca34042a6b90fdb6e2691eefde6e8516000cb4faa89a53a6e968a040917"} Jul 13 16:39:47.731307 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.731267 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 16:34:47 +0000 UTC" deadline="2027-12-31 21:37:07.748618771 +0000 UTC" Jul 13 16:39:47.731366 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:47.731307 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12868h57m20.017314186s" Jul 13 16:39:47.749192 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.749170 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8647d58d_2479_44a1_892d_4ee82ebcf4ef.slice/crio-084c6caa67a61d2865bf4ee976d518dd39f7a8654632e9c5a45c63e365b48f1b WatchSource:0}: Error finding container 084c6caa67a61d2865bf4ee976d518dd39f7a8654632e9c5a45c63e365b48f1b: Status 404 returned error can't find the container with id 084c6caa67a61d2865bf4ee976d518dd39f7a8654632e9c5a45c63e365b48f1b Jul 13 16:39:47.919564 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.919530 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d19c188_a7a4_415e_b9ae_2446e3399c85.slice/crio-8e5b38f6d9ea28880e766f6f53ed9e1fe1e3b71ec6c645a5b1f1574611d87cef WatchSource:0}: Error finding container 8e5b38f6d9ea28880e766f6f53ed9e1fe1e3b71ec6c645a5b1f1574611d87cef: Status 404 returned error can't find the container with id 8e5b38f6d9ea28880e766f6f53ed9e1fe1e3b71ec6c645a5b1f1574611d87cef Jul 13 16:39:47.946660 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:39:47.946631 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b20c96_8862_45d6_882e_d20a22dba057.slice/crio-be91c5afa744645738267c95a19303c2d57314d8c00e406b3dfa40129e58a69a WatchSource:0}: Error finding container be91c5afa744645738267c95a19303c2d57314d8c00e406b3dfa40129e58a69a: Status 404 returned error can't find the container with id be91c5afa744645738267c95a19303c2d57314d8c00e406b3dfa40129e58a69a Jul 13 16:39:48.282458 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.282377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:48.282458 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.282434 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:48.282687 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.282590 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:48.282687 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.282653 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:50.282634347 +0000 UTC m=+5.072722270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:48.283148 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.283067 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:48.283148 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.283141 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:50.283108463 +0000 UTC m=+5.073196398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:48.383319 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.383277 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:48.383682 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.383654 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:39:48.383682 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.383681 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:39:48.383845 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.383696 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:48.383845 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.383768 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:39:50.383739016 +0000 UTC m=+5.173826935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:48.714776 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.714661 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:48.715287 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.714800 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:48.715646 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.715620 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:48.715845 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:48.715814 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:48.731990 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.731903 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 16:34:47 +0000 UTC" deadline="2028-02-29 12:01:51.523698601 +0000 UTC" Jul 13 16:39:48.731990 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.731967 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14299h22m2.791735689s" Jul 13 16:39:48.771276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.771233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5zv66" event={"ID":"f6b20c96-8862-45d6-882e-d20a22dba057","Type":"ContainerStarted","Data":"be91c5afa744645738267c95a19303c2d57314d8c00e406b3dfa40129e58a69a"} Jul 13 16:39:48.782500 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.782464 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qjvcv" event={"ID":"8647d58d-2479-44a1-892d-4ee82ebcf4ef","Type":"ContainerStarted","Data":"084c6caa67a61d2865bf4ee976d518dd39f7a8654632e9c5a45c63e365b48f1b"} Jul 13 16:39:48.796864 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:48.796828 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"8e5b38f6d9ea28880e766f6f53ed9e1fe1e3b71ec6c645a5b1f1574611d87cef"} Jul 13 16:39:49.716584 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:49.716548 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:49.717067 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:49.716710 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:50.304044 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:50.304002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:50.304266 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:50.304085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:50.304266 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.304219 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:50.304389 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.304281 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:54.304260114 +0000 UTC m=+9.094348035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:50.304728 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.304679 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:50.304821 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.304732 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:39:54.30471619 +0000 UTC m=+9.094804122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:50.405060 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:50.404967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:50.405309 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.405255 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:39:50.405309 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.405285 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:39:50.405309 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.405299 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:50.405490 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.405367 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:39:54.405347055 +0000 UTC m=+9.195434971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:50.714767 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:50.714060 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:50.714767 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.714228 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:50.714767 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:50.714072 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:50.714767 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:50.714720 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:51.714905 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:51.714870 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:51.715639 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:51.715012 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:52.714460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:52.714351 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:52.714460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:52.714394 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:52.714718 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:52.714482 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:52.714924 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:52.714873 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:53.715955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:53.714395 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:53.715955 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:53.714537 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:54.339776 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:54.339122 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:54.339776 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:54.339224 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:54.339776 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.339356 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:54.339776 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.339418 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:02.339400125 +0000 UTC m=+17.129488042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:39:54.339776 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.339748 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:54.340232 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.339827 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:02.339808173 +0000 UTC m=+17.129896096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:39:54.440402 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:54.440319 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:54.440562 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.440478 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:39:54.440562 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.440502 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:39:54.440562 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.440515 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:54.440720 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.440570 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:40:02.440552283 +0000 UTC m=+17.230640210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:39:54.714507 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:54.713890 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:54.714507 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.714021 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:54.714507 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:54.714460 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:54.714949 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:54.714561 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:55.715304 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:55.715096 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:55.715304 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:55.715213 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:56.714906 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:56.714868 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:56.715123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:56.714865 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:56.715200 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:56.715135 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:56.715200 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:56.714996 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:57.714072 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:57.713948 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:57.714507 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:57.714097 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:39:58.714189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:58.713986 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:39:58.714612 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:58.713995 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:39:58.714612 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:58.714283 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:39:58.714612 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:58.714335 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:39:59.714339 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:39:59.714262 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:39:59.714736 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:39:59.714410 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:00.714770 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:00.714733 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:00.715244 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:00.714733 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:00.715244 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:00.714856 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:00.715244 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:00.714943 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:01.714949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:01.714912 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:01.715465 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:01.715067 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:02.398003 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:02.397963 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:02.398168 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:02.398016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:02.398168 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.398132 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:40:02.398281 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.398197 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:18.398182329 +0000 UTC m=+33.188270245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:40:02.398281 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.398132 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:40:02.398281 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.398236 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:18.398226837 +0000 UTC m=+33.188314751 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:40:02.499275 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:02.499231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:02.499434 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.499399 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:40:02.499434 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.499419 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:40:02.499434 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.499430 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:40:02.499570 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.499487 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:40:18.499467375 +0000 UTC m=+33.289555307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:40:02.714151 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:02.714069 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:02.714151 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:02.714085 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:02.714347 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.714187 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:02.714347 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:02.714325 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:03.714846 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:03.714761 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:03.715314 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:03.714900 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:04.714346 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:04.714318 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:04.714525 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:04.714359 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:04.714525 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:04.714430 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:04.714632 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:04.714552 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:05.715382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:05.715349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:05.715818 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:05.715474 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:06.714908 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:06.714874 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:06.715135 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:06.714874 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:06.715135 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:06.715005 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:06.715135 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:06.715104 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:07.714991 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:07.714950 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:07.715433 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:07.715104 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:08.714270 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.714002 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:08.714373 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.714060 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:08.714409 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:08.714368 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:08.714443 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:08.714414 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:08.848264 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848238 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848575 2569 generic.go:356] "Generic (PLEG): container finished" podID="5d19c188-a7a4-415e-b9ae-2446e3399c85" containerID="f7cfcb29015342630f04c1b033e43ae7d732649d2344fb4ffe737545d87bc2fb" exitCode=1 Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848646 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"7f00e927fc1af89267a908acdba8d0227a4d9ab72cf922011e04855281ae47fb"} Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848709 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"eef666e6070119252242174c56081f1e72d3653ade181364f4e79fdf1f4529ed"} Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848750 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"5256e9fe70fde4692abdedf9f3115c333e4130671915ef0bc23e0382ee828193"} Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848779 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"c88f9303f57723fae9f8847c0b574526565f3420d0dab2d1028bb9eab56cf058"} Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848807 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerDied","Data":"f7cfcb29015342630f04c1b033e43ae7d732649d2344fb4ffe737545d87bc2fb"} Jul 13 16:40:08.848916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.848850 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"75bc4c6226e7afe5468e525ba431c725c3932f87e158699efaf428efe9b4a99d"} Jul 13 16:40:08.850119 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.850100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" event={"ID":"10036d42-4c47-4162-afb3-a04d6c213a01","Type":"ContainerStarted","Data":"89d285d43a96fc9295587e6117eb354105e42b718b8c8298d1e78a4b07e5e6c0"} Jul 13 16:40:08.851431 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.851412 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2ppc" event={"ID":"4cc7aca3-ab5d-4c42-af22-45450b82be31","Type":"ContainerStarted","Data":"c35d57697c1ba15c6fbe8374675a8ff38afbf7c8469ce140378d0f78fdf3706c"} Jul 13 16:40:08.852717 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.852697 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5zv66" event={"ID":"f6b20c96-8862-45d6-882e-d20a22dba057","Type":"ContainerStarted","Data":"70527de94c5f73718ba518061245d40b77ba9156dfc5ee166dd149393c2e562a"} Jul 13 16:40:08.854136 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.854115 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="4c1e86ae7f945fc6a08340b5991c6b858f592b733900f5fd3978bf25dc50463d" exitCode=0 Jul 13 16:40:08.854201 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.854176 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"4c1e86ae7f945fc6a08340b5991c6b858f592b733900f5fd3978bf25dc50463d"} Jul 13 16:40:08.855628 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.855604 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7m9tz" event={"ID":"c5b7a469-647c-4afe-91a5-e11edfb96827","Type":"ContainerStarted","Data":"558bab78aa0c1e9f3d5918574b90f00aab6e46478aa86619e91bff01b6be0ef6"} Jul 13 16:40:08.857049 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.857010 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbzzc" event={"ID":"b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f","Type":"ContainerStarted","Data":"eb35c22a2d57c331f867c00c2c0049f337f72cd62cee156928490fdb58de99cc"} Jul 13 16:40:08.858293 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.858273 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2jlz5" event={"ID":"163640c8-2bec-4876-a451-71fd060c9c9f","Type":"ContainerStarted","Data":"a30e55c2aefed769d7207bf8914826f29b616afa54b8fc2f111afaac34ba3d85"} Jul 13 16:40:08.859631 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.859610 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" event={"ID":"1c9b46fdf787e2321bc5c8391e702436","Type":"ContainerStarted","Data":"3ab0609f107c8cb1f26a93de533b2f997cf142ecb49d19a2cc06e867a2774ea8"} Jul 13 16:40:08.860920 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.860902 2569 generic.go:356] "Generic (PLEG): container finished" podID="f270f0b2772721148249b750c246fb3f" containerID="e899a225e7be758f555f8346c60cb0134418f8724cf47142b5a01a8369f5cf66" exitCode=0 Jul 13 16:40:08.860993 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.860927 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" event={"ID":"f270f0b2772721148249b750c246fb3f","Type":"ContainerDied","Data":"e899a225e7be758f555f8346c60cb0134418f8724cf47142b5a01a8369f5cf66"} Jul 13 16:40:08.867417 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.867375 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m2ppc" podStartSLOduration=2.631846118 podStartE2EDuration="22.86736441s" totalImagesPullingTime="20.235518292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.866723278 +0000 UTC m=+23.656811213" watchObservedRunningTime="2026-07-13 16:40:08.86736441 +0000 UTC m=+23.657452335" Jul 13 16:40:08.880025 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.879987 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-178.ec2.internal" podStartSLOduration=22.879976636 podStartE2EDuration="22.879976636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.879764676 +0000 UTC m=+23.669852610" watchObservedRunningTime="2026-07-13 16:40:08.879976636 +0000 UTC m=+23.670064570" Jul 13 16:40:08.910207 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.910168 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nbzzc" podStartSLOduration=2.688393823 podStartE2EDuration="22.91015489s" totalImagesPullingTime="20.221761067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.896490525 +0000 UTC m=+23.686578459" watchObservedRunningTime="2026-07-13 16:40:08.91015489 +0000 UTC m=+23.700242824" Jul 13 16:40:08.910321 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.910302 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7m9tz" podStartSLOduration=2.661366076 podStartE2EDuration="22.910297491s" totalImagesPullingTime="20.248931415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.909801729 +0000 UTC m=+23.699889674" watchObservedRunningTime="2026-07-13 16:40:08.910297491 +0000 UTC m=+23.700385455" Jul 13 16:40:08.925970 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.925882 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2jlz5" podStartSLOduration=2.57975922 podStartE2EDuration="22.925869973s" totalImagesPullingTime="20.346110753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.925708543 +0000 UTC m=+23.715796477" watchObservedRunningTime="2026-07-13 16:40:08.925869973 +0000 UTC m=+23.715957907" Jul 13 16:40:08.948388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:08.948355 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5zv66" podStartSLOduration=3.178762801 podStartE2EDuration="22.948343678s" totalImagesPullingTime="19.769580877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:08.948338942 +0000 UTC m=+23.738426875" watchObservedRunningTime="2026-07-13 16:40:08.948343678 +0000 UTC m=+23.738431611" Jul 13 16:40:09.714882 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:09.714817 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:09.715137 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:09.714923 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:09.864519 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:09.864488 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qjvcv" event={"ID":"8647d58d-2479-44a1-892d-4ee82ebcf4ef","Type":"ContainerStarted","Data":"5832bab6d7c3b0fe0532f1c1ac7180441e7e05342b1d310879f6f35083181ea6"} Jul 13 16:40:09.893972 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:09.893916 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qjvcv" podStartSLOduration=3.92194441 podStartE2EDuration="23.893899298s" totalImagesPullingTime="19.971954888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:09.893439295 +0000 UTC m=+24.683527230" watchObservedRunningTime="2026-07-13 16:40:09.893899298 +0000 UTC m=+24.683987231" Jul 13 16:40:09.896611 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:09.896569 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 16:40:10.267162 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.267086 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:40:10.267869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.267850 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:40:10.702678 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.702503 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T16:40:09.896586582Z","UUID":"be9941d0-09e2-4f4d-bf36-a16f74f65839","Handler":null,"Name":"","Endpoint":""} Jul 13 16:40:10.704327 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.704301 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 16:40:10.704468 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.704343 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 16:40:10.714708 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.714647 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:10.714873 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:10.714762 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:10.714873 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.714833 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:10.714992 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:10.714952 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:10.869557 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.869514 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" event={"ID":"10036d42-4c47-4162-afb3-a04d6c213a01","Type":"ContainerStarted","Data":"77a0eec24e4aa0f077aa8a486d5c8191e92addec326b946fa325ee049369fc07"} Jul 13 16:40:10.871815 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.871741 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" event={"ID":"f270f0b2772721148249b750c246fb3f","Type":"ContainerStarted","Data":"c6a4f651a26cc83f332469a782b1400de4ede543ab7bd66fcfc4f8a0a8144bc8"} Jul 13 16:40:10.888595 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:10.888533 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-178.ec2.internal" podStartSLOduration=24.888514786000002 podStartE2EDuration="24.888514786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:10.888163002 +0000 UTC m=+25.678250937" watchObservedRunningTime="2026-07-13 16:40:10.888514786 +0000 UTC m=+25.678602721" Jul 13 16:40:11.714701 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.714471 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:11.714856 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:11.714742 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:11.876265 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.876240 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:40:11.876774 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.876549 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"56e5924ed56e7a0b01302e5067bea5ec357a3d47d95522896fd375002f87e2d7"} Jul 13 16:40:11.878318 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.878293 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" event={"ID":"10036d42-4c47-4162-afb3-a04d6c213a01","Type":"ContainerStarted","Data":"45375c85583269bc840d48fdc9cbf7e86a6dbb6e8e45aee14eecb05e1a27f768"} Jul 13 16:40:11.878318 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.878311 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 16:40:11.896638 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:11.896596 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sfzdx" podStartSLOduration=3.05887809 podStartE2EDuration="25.896583446s" totalImagesPullingTime="22.837705356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:11.895973328 +0000 UTC m=+26.686061262" watchObservedRunningTime="2026-07-13 16:40:11.896583446 +0000 UTC m=+26.686671380" Jul 13 16:40:12.714231 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:12.714196 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:12.714412 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:12.714198 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:12.714412 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:12.714326 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:12.714412 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:12.714369 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:13.013407 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.013327 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:40:13.013930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.013431 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 16:40:13.013930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.013860 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nbzzc" Jul 13 16:40:13.714314 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.714153 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:13.714464 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:13.714424 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:13.885761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.885739 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:40:13.886134 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.886111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"c36b922d62c4dc01d56d8928c9090eeb05872601365959050167241312177708"} Jul 13 16:40:13.886385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.886339 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:13.886526 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.886509 2569 scope.go:117] "RemoveContainer" containerID="f7cfcb29015342630f04c1b033e43ae7d732649d2344fb4ffe737545d87bc2fb" Jul 13 16:40:13.888338 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.888318 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="bcc060e175addc5324326f6655879dd7722f79e3c8ea340d58ad9bfa3d38facf" exitCode=0 Jul 13 16:40:13.888431 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.888405 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"bcc060e175addc5324326f6655879dd7722f79e3c8ea340d58ad9bfa3d38facf"} Jul 13 16:40:13.902653 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:13.902633 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:14.714718 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.714676 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:14.715433 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:14.714799 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:14.715433 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.714677 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:14.715433 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:14.714998 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:14.894297 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.894269 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:40:14.894650 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.894630 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" event={"ID":"5d19c188-a7a4-415e-b9ae-2446e3399c85","Type":"ContainerStarted","Data":"81dacd0df7f1c1d7db862a1b9738cf410895098b2dc8729a7c0e172f0a0c5c18"} Jul 13 16:40:14.894941 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.894912 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:14.895020 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.894980 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:14.910746 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.910725 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:14.990573 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:14.990482 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" podStartSLOduration=8.927352254 podStartE2EDuration="28.990468082s" totalImagesPullingTime="20.063115828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:14.953293801 +0000 UTC m=+29.743381734" watchObservedRunningTime="2026-07-13 16:40:14.990468082 +0000 UTC m=+29.780556015" Jul 13 16:40:15.714703 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.714621 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:15.714957 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:15.714722 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:15.898654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.898623 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="1e93060ab6c1837dcd97a884c217b747c67554bbec714961fc5c23b221bd11ea" exitCode=0 Jul 13 16:40:15.898801 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.898708 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"1e93060ab6c1837dcd97a884c217b747c67554bbec714961fc5c23b221bd11ea"} Jul 13 16:40:15.979726 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.979666 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4r5xc"] Jul 13 16:40:15.979883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.979752 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:15.979883 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:15.979838 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:15.983780 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.983744 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-grd9b"] Jul 13 16:40:15.983921 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.983892 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:15.984020 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:15.984001 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:15.984406 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.984386 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-254w7"] Jul 13 16:40:15.984523 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:15.984474 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:15.984594 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:15.984572 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:17.714196 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:17.714015 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:17.714627 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:17.714076 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:17.714627 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:17.714316 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:17.714627 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:17.714104 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:17.714627 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:17.714400 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:17.714627 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:17.714475 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:17.906156 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:17.906127 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="bbe64efa0e2915f9c5cefc30d08b37fb84669bd3c2cd88089db4766d2a55cadf" exitCode=0 Jul 13 16:40:17.906307 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:17.906206 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"bbe64efa0e2915f9c5cefc30d08b37fb84669bd3c2cd88089db4766d2a55cadf"} Jul 13 16:40:18.418081 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:18.418046 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:18.418260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:18.418117 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:18.418260 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.418133 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:40:18.418260 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.418194 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:50.418175396 +0000 UTC m=+65.208263315 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 16:40:18.418260 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.418201 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 16:40:18.418260 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.418236 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret podName:81257490-18b6-48fa-8601-eef4639687e1 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:50.418224493 +0000 UTC m=+65.208312405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret") pod "global-pull-secret-syncer-grd9b" (UID: "81257490-18b6-48fa-8601-eef4639687e1") : object "kube-system"/"original-pull-secret" not registered Jul 13 16:40:18.519214 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:18.519187 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:18.519356 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.519319 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 16:40:18.519356 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.519335 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 16:40:18.519356 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.519344 2569 projected.go:196] Error preparing data for projected volume kube-api-access-dbxjq for pod openshift-network-diagnostics/network-check-target-254w7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:40:18.519456 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:18.519389 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq podName:502b550d-c0f8-4cf7-961e-f8c9aa8fff9e nodeName:}" failed. No retries permitted until 2026-07-13 16:40:50.519375757 +0000 UTC m=+65.309463672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dbxjq" (UniqueName: "kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq") pod "network-check-target-254w7" (UID: "502b550d-c0f8-4cf7-961e-f8c9aa8fff9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 16:40:19.714350 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:19.714317 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:19.715016 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:19.714317 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:19.715016 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:19.714434 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:19.715016 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:19.714516 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:19.715016 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:19.714319 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:19.715016 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:19.714630 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:21.714490 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.714451 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:21.715084 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.714451 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:21.715084 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:21.714592 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-254w7" podUID="502b550d-c0f8-4cf7-961e-f8c9aa8fff9e" Jul 13 16:40:21.715084 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.714458 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:21.715084 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:21.714654 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4r5xc" podUID="a2326b3b-3b91-4d9f-957b-5a0cd8e78573" Jul 13 16:40:21.715084 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:21.714756 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-grd9b" podUID="81257490-18b6-48fa-8601-eef4639687e1" Jul 13 16:40:21.964246 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.964217 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-178.ec2.internal" event="NodeReady" Jul 13 16:40:21.964403 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.964367 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 16:40:21.997153 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:21.997077 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:40:22.032945 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.032766 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zftr5"] Jul 13 16:40:22.032945 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.032899 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.035808 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.035784 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 16:40:22.035808 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.035816 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 16:40:22.036073 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.035786 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 16:40:22.036159 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.036140 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6ng2j\"" Jul 13 16:40:22.041730 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.041711 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 16:40:22.045432 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045411 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:40:22.045540 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045447 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w9b2j"] Jul 13 16:40:22.045540 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045527 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.045656 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045722 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045693 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045774 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045723 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045774 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045766 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045870 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045830 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045929 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045988 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.045988 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.045966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxhs7\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.048069 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.048030 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 16:40:22.048155 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.048033 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mgdqr\"" Jul 13 16:40:22.048155 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.048057 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 16:40:22.061567 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.061547 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zftr5"] Jul 13 16:40:22.061673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.061586 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9b2j"] Jul 13 16:40:22.061673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.061663 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.064374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.064355 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 16:40:22.064476 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.064382 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 16:40:22.064476 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.064395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 16:40:22.064595 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.064483 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-76zsq\"" Jul 13 16:40:22.146627 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146598 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.146821 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxhs7\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.146821 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146683 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbpw\" (UniqueName: \"kubernetes.io/projected/25174820-93a8-4044-8154-889410fec806-kube-api-access-xwbpw\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.146821 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146705 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267f22ff-d752-4298-923c-5baa9e0ad6e6-tmp-dir\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.146821 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146729 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.146821 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146798 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdlq\" (UniqueName: \"kubernetes.io/projected/267f22ff-d752-4298-923c-5baa9e0ad6e6-kube-api-access-zpdlq\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146923 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.146976 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25174820-93a8-4044-8154-889410fec806-cert\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147008 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147063 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147111 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147090 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/267f22ff-d752-4298-923c-5baa9e0ad6e6-config-volume\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.147430 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147120 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147430 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.147205 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:22.147430 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.147219 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:22.147430 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.147268 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:22.64725477 +0000 UTC m=+37.437342682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:22.147430 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147332 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.147704 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.147678 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.148328 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.148308 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.151514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.151377 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.151616 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.151393 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.158118 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.158094 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxhs7\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.158209 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.158167 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.160411 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.160384 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nbzzc_b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f/konnectivity-agent/0.log" limit=1024 Jul 13 16:40:22.248471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248393 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbpw\" (UniqueName: \"kubernetes.io/projected/25174820-93a8-4044-8154-889410fec806-kube-api-access-xwbpw\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.248471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248428 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267f22ff-d752-4298-923c-5baa9e0ad6e6-tmp-dir\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.248471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248445 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.248471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248469 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdlq\" (UniqueName: \"kubernetes.io/projected/267f22ff-d752-4298-923c-5baa9e0ad6e6-kube-api-access-zpdlq\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.248795 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.248574 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:22.248795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25174820-93a8-4044-8154-889410fec806-cert\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.248795 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.248698 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:22.748676091 +0000 UTC m=+37.538764020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:22.248795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/267f22ff-d752-4298-923c-5baa9e0ad6e6-config-volume\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.248795 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.248782 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267f22ff-d752-4298-923c-5baa9e0ad6e6-tmp-dir\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.249260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.249241 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/267f22ff-d752-4298-923c-5baa9e0ad6e6-config-volume\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.251014 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.250995 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25174820-93a8-4044-8154-889410fec806-cert\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.257327 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.257307 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdlq\" (UniqueName: \"kubernetes.io/projected/267f22ff-d752-4298-923c-5baa9e0ad6e6-kube-api-access-zpdlq\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.257413 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.257389 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbpw\" (UniqueName: \"kubernetes.io/projected/25174820-93a8-4044-8154-889410fec806-kube-api-access-xwbpw\") pod \"ingress-canary-w9b2j\" (UID: \"25174820-93a8-4044-8154-889410fec806\") " pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.266819 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.266785 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nbzzc_b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f/konnectivity-agent/0.log" limit=1024 Jul 13 16:40:22.371152 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.371116 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9b2j" Jul 13 16:40:22.536054 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.536010 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9b2j"] Jul 13 16:40:22.540282 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:22.540260 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25174820_93a8_4044_8154_889410fec806.slice/crio-34aa50e13695d8cb05fe03ffd88b80c83c7570f85394ef5b30c442295ce3e7b5 WatchSource:0}: Error finding container 34aa50e13695d8cb05fe03ffd88b80c83c7570f85394ef5b30c442295ce3e7b5: Status 404 returned error can't find the container with id 34aa50e13695d8cb05fe03ffd88b80c83c7570f85394ef5b30c442295ce3e7b5 Jul 13 16:40:22.652534 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.652497 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:22.652665 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.652648 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:22.652705 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.652667 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:22.652742 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.652723 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:23.652708369 +0000 UTC m=+38.442796281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:22.753096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.753063 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:22.753467 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.753201 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:22.753467 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:22.753261 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:23.75324349 +0000 UTC m=+38.543331402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:22.919655 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.919624 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9b2j" event={"ID":"25174820-93a8-4044-8154-889410fec806","Type":"ContainerStarted","Data":"34aa50e13695d8cb05fe03ffd88b80c83c7570f85394ef5b30c442295ce3e7b5"} Jul 13 16:40:22.922418 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.922388 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="5c749451ce6b31b3e2610b4d6b0eff27577c7ee8f8f2e1169f4bbc295f619fff" exitCode=0 Jul 13 16:40:22.922525 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:22.922435 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"5c749451ce6b31b3e2610b4d6b0eff27577c7ee8f8f2e1169f4bbc295f619fff"} Jul 13 16:40:23.660221 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.660185 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:23.660421 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:23.660324 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:23.660421 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:23.660341 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:23.660421 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:23.660393 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:25.660379768 +0000 UTC m=+40.450467684 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:23.717901 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.717873 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:23.718070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.717873 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:23.718070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.717877 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:23.723540 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.723511 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 16:40:23.723540 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.723511 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79qpb\"" Jul 13 16:40:23.723725 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.723511 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 16:40:23.724305 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.724285 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 16:40:23.724405 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.724289 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 16:40:23.724405 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.724365 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-whvdd\"" Jul 13 16:40:23.761478 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.761455 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:23.761870 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:23.761585 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:23.761870 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:23.761649 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:25.761631934 +0000 UTC m=+40.551719855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:23.929338 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.929255 2569 generic.go:356] "Generic (PLEG): container finished" podID="72abcb4e-bd9b-4846-85c4-132f6a46fabf" containerID="40073b9299f6ca9bc743dc6bf1c8b68299752e59d2110b25ec060d4b7a772b53" exitCode=0 Jul 13 16:40:23.929338 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:23.929299 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerDied","Data":"40073b9299f6ca9bc743dc6bf1c8b68299752e59d2110b25ec060d4b7a772b53"} Jul 13 16:40:24.935888 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:24.935679 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" event={"ID":"72abcb4e-bd9b-4846-85c4-132f6a46fabf","Type":"ContainerStarted","Data":"0765e470f92c1a32a97653b10417312a5e30b2f866fe22b3f1cf046bf9face1d"} Jul 13 16:40:24.965679 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:24.965580 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rw5cq" podStartSLOduration=6.883057154 podStartE2EDuration="38.965558633s" totalImagesPullingTime="32.082501479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:24.961387566 +0000 UTC m=+39.751475499" watchObservedRunningTime="2026-07-13 16:40:24.965558633 +0000 UTC m=+39.755646568" Jul 13 16:40:25.091907 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.091872 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nbzzc_b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f/konnectivity-agent/0.log" limit=1024 Jul 13 16:40:25.147504 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.147451 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nbzzc_b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f/konnectivity-agent/0.log" limit=1024 Jul 13 16:40:25.680863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.680831 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:25.681007 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:25.680947 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:25.681007 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:25.680959 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:25.681007 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:25.681006 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.680994038 +0000 UTC m=+44.471081950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:25.781577 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.781545 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:25.781724 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:25.781679 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:25.781767 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:25.781729 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.781715348 +0000 UTC m=+44.571803262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:25.939293 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.939216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9b2j" event={"ID":"25174820-93a8-4044-8154-889410fec806","Type":"ContainerStarted","Data":"7cfa53b435380553a20dfd0d37982a7a7ed55380ff90cc51cfb2e440037a8ef8"} Jul 13 16:40:25.957374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:25.957335 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w9b2j" podStartSLOduration=2.09736968 podStartE2EDuration="4.957319401s" totalImagesPullingTime="2.859949721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:25.95691724 +0000 UTC m=+40.747005188" watchObservedRunningTime="2026-07-13 16:40:25.957319401 +0000 UTC m=+40.747407335" Jul 13 16:40:27.892344 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.891196 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl"] Jul 13 16:40:27.910548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.910515 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f"] Jul 13 16:40:27.910688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.910636 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:27.913371 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.913346 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:27.913371 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.913346 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 16:40:27.913554 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.913386 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8pkvw\"" Jul 13 16:40:27.913554 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.913356 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 16:40:27.913554 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.913452 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 16:40:27.935840 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.935815 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7657f6f484-wcq5g"] Jul 13 16:40:27.935967 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.935946 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" Jul 13 16:40:27.938271 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.938252 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-64ngq\"" Jul 13 16:40:27.957986 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.957964 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl"] Jul 13 16:40:27.958101 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.957991 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7657f6f484-wcq5g"] Jul 13 16:40:27.958101 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.958004 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f"] Jul 13 16:40:27.958211 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.958101 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:27.961053 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961014 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 16:40:27.961140 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961070 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 16:40:27.961140 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961073 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 16:40:27.961140 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961096 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 16:40:27.961140 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961111 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 16:40:27.961140 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961112 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 16:40:27.961367 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.961076 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5s6dv\"" Jul 13 16:40:27.993710 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.993687 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf"] Jul 13 16:40:27.995183 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:27.995253 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:27.995253 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995209 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:27.995253 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995230 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gktnq\" (UniqueName: \"kubernetes.io/projected/1d1f00f5-f46d-4962-8bfb-c49decad5994-kube-api-access-gktnq\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:27.995253 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kzcd\" (UniqueName: \"kubernetes.io/projected/1406b058-b79d-45d3-9a4d-2f0a3012f7ff-kube-api-access-6kzcd\") pod \"network-check-source-869dcfcdcd-ksk5f\" (UID: \"1406b058-b79d-45d3-9a4d-2f0a3012f7ff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" Jul 13 16:40:27.995449 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995400 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-stats-auth\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:27.995449 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995431 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:27.995527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995467 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jp2v\" (UniqueName: \"kubernetes.io/projected/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-kube-api-access-8jp2v\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:27.995527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:27.995517 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-default-certificate\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.010001 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.009987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" Jul 13 16:40:28.012757 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.012732 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-kmkjt"] Jul 13 16:40:28.013471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.013450 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.014275 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.014259 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6rvtx\"" Jul 13 16:40:28.014931 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.014830 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.036011 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.035990 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld"] Jul 13 16:40:28.036137 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.036120 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.040075 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.040056 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 13 16:40:28.040075 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.040067 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.040232 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.040066 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tfbwf\"" Jul 13 16:40:28.040232 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.040058 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 13 16:40:28.040353 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.040341 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.045115 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.045098 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 13 16:40:28.054050 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.054020 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q"] Jul 13 16:40:28.054157 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.054134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.056403 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.056377 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.056477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.056463 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 16:40:28.056599 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.056588 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 16:40:28.056657 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.056641 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.056705 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.056669 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9dnsc\"" Jul 13 16:40:28.076124 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.076102 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt"] Jul 13 16:40:28.076227 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.076215 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.078685 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.078661 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.078772 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.078717 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 16:40:28.078772 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.078746 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j2nxs\"" Jul 13 16:40:28.079066 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.079052 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.096734 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096709 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kzcd\" (UniqueName: \"kubernetes.io/projected/1406b058-b79d-45d3-9a4d-2f0a3012f7ff-kube-api-access-6kzcd\") pod \"network-check-source-869dcfcdcd-ksk5f\" (UID: \"1406b058-b79d-45d3-9a4d-2f0a3012f7ff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" Jul 13 16:40:28.096820 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096751 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.096820 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096780 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb9f57-2e10-4550-b60f-6d7956a460ea-serving-cert\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.096930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096848 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89fnd\" (UniqueName: \"kubernetes.io/projected/680fabb0-064a-4fc1-89f9-53d8c010bdd0-kube-api-access-89fnd\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.096930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/680fabb0-064a-4fc1-89f9-53d8c010bdd0-config\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.096930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.096933 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.096968 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-trusted-ca\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.096997 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:28.596983372 +0000 UTC m=+43.387071289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097014 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jp2v\" (UniqueName: \"kubernetes.io/projected/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-kube-api-access-8jp2v\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097031 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-config\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097077 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-stats-auth\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.097085 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.097085 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:28.597070512 +0000 UTC m=+43.387158425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:28.097460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097214 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4kbd\" (UniqueName: \"kubernetes.io/projected/d5cb9f57-2e10-4550-b60f-6d7956a460ea-kube-api-access-j4kbd\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.097460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097316 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5jhs\" (UniqueName: \"kubernetes.io/projected/67d6774c-170b-4704-8f96-3eab4de7fbf1-kube-api-access-t5jhs\") pod \"volume-data-source-validator-78495597f6-4tqkf\" (UID: \"67d6774c-170b-4704-8f96-3eab4de7fbf1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" Jul 13 16:40:28.097460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-default-certificate\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.097460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/680fabb0-064a-4fc1-89f9-53d8c010bdd0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.097460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.097674 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.097485 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gktnq\" (UniqueName: \"kubernetes.io/projected/1d1f00f5-f46d-4962-8bfb-c49decad5994-kube-api-access-gktnq\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.098059 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098008 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098310 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf"] Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098343 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld"] Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098362 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kmkjt"] Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098381 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q"] Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098403 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt"] Jul 13 16:40:28.098451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098431 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.098736 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.098434 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wjqdg"] Jul 13 16:40:28.100855 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.100836 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 16:40:28.101175 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.101155 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lc7lk\"" Jul 13 16:40:28.101354 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.101336 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.101455 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.101438 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-stats-auth\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.101624 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.101601 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-default-certificate\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.102217 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.102205 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.102276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.102237 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.102276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.102250 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 16:40:28.120533 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.120512 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7rwks"] Jul 13 16:40:28.120638 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.120622 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.123063 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.123018 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jp2v\" (UniqueName: \"kubernetes.io/projected/f6a72c4e-2845-4db9-9cae-ee53a2a8daad-kube-api-access-8jp2v\") pod \"kube-storage-version-migrator-operator-799cb7f958-2n6wl\" (UID: \"f6a72c4e-2845-4db9-9cae-ee53a2a8daad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.123260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.123241 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 16:40:28.123318 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.123271 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 16:40:28.123367 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.123245 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-655cv\"" Jul 13 16:40:28.129061 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.129024 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gktnq\" (UniqueName: \"kubernetes.io/projected/1d1f00f5-f46d-4962-8bfb-c49decad5994-kube-api-access-gktnq\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.129601 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.129583 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kzcd\" (UniqueName: \"kubernetes.io/projected/1406b058-b79d-45d3-9a4d-2f0a3012f7ff-kube-api-access-6kzcd\") pod \"network-check-source-869dcfcdcd-ksk5f\" (UID: \"1406b058-b79d-45d3-9a4d-2f0a3012f7ff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" Jul 13 16:40:28.144739 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.144723 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wjqdg"] Jul 13 16:40:28.144836 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.144751 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7rwks"] Jul 13 16:40:28.144836 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.144830 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.147378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.147358 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 16:40:28.147464 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.147433 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 16:40:28.147464 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.147443 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zlpjz\"" Jul 13 16:40:28.147581 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.147521 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 16:40:28.147581 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.147550 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 16:40:28.153130 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.153113 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 16:40:28.197881 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.197858 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4kbd\" (UniqueName: \"kubernetes.io/projected/d5cb9f57-2e10-4550-b60f-6d7956a460ea-kube-api-access-j4kbd\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.197975 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.197884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5jhs\" (UniqueName: \"kubernetes.io/projected/67d6774c-170b-4704-8f96-3eab4de7fbf1-kube-api-access-t5jhs\") pod \"volume-data-source-validator-78495597f6-4tqkf\" (UID: \"67d6774c-170b-4704-8f96-3eab4de7fbf1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" Jul 13 16:40:28.197975 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.197915 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/680fabb0-064a-4fc1-89f9-53d8c010bdd0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.197975 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.197936 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/816d33a9-20d0-440b-a9f3-914311397007-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.197975 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.197965 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7425c4b6-381c-445f-828e-98d197d6d0c6-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.198169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198006 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.198169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198056 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdxpq\" (UniqueName: \"kubernetes.io/projected/7425c4b6-381c-445f-828e-98d197d6d0c6-kube-api-access-tdxpq\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.198169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-serving-cert\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198154 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb9f57-2e10-4550-b60f-6d7956a460ea-serving-cert\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198183 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89fnd\" (UniqueName: \"kubernetes.io/projected/680fabb0-064a-4fc1-89f9-53d8c010bdd0-kube-api-access-89fnd\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198199 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/680fabb0-064a-4fc1-89f9-53d8c010bdd0-config\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198214 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-snapshots\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198259 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198294 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkkrq\" (UniqueName: \"kubernetes.io/projected/1b07bb5f-3052-47ea-afae-018bb50a9e43-kube-api-access-fkkrq\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.198326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198323 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.198645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198396 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7lcg\" (UniqueName: \"kubernetes.io/projected/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-kube-api-access-p7lcg\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198438 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-trusted-ca\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.198645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198466 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-tmp\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198497 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.198645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-config\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.198878 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.198688 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/680fabb0-064a-4fc1-89f9-53d8c010bdd0-config\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.199086 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.199055 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-config\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.199258 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.199240 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5cb9f57-2e10-4550-b60f-6d7956a460ea-trusted-ca\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.200285 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.200267 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cb9f57-2e10-4550-b60f-6d7956a460ea-serving-cert\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.200347 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.200297 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/680fabb0-064a-4fc1-89f9-53d8c010bdd0-serving-cert\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.206413 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.206388 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89fnd\" (UniqueName: \"kubernetes.io/projected/680fabb0-064a-4fc1-89f9-53d8c010bdd0-kube-api-access-89fnd\") pod \"service-ca-operator-7695fc6c8c-thtld\" (UID: \"680fabb0-064a-4fc1-89f9-53d8c010bdd0\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.206794 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.206777 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5jhs\" (UniqueName: \"kubernetes.io/projected/67d6774c-170b-4704-8f96-3eab4de7fbf1-kube-api-access-t5jhs\") pod \"volume-data-source-validator-78495597f6-4tqkf\" (UID: \"67d6774c-170b-4704-8f96-3eab4de7fbf1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" Jul 13 16:40:28.207305 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.207290 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4kbd\" (UniqueName: \"kubernetes.io/projected/d5cb9f57-2e10-4550-b60f-6d7956a460ea-kube-api-access-j4kbd\") pod \"console-operator-7866497494-kmkjt\" (UID: \"d5cb9f57-2e10-4550-b60f-6d7956a460ea\") " pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.220114 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.220096 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" Jul 13 16:40:28.243533 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.243512 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299283 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7lcg\" (UniqueName: \"kubernetes.io/projected/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-kube-api-access-p7lcg\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-tmp\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299373 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/816d33a9-20d0-440b-a9f3-914311397007-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299476 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7425c4b6-381c-445f-828e-98d197d6d0c6-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdxpq\" (UniqueName: \"kubernetes.io/projected/7425c4b6-381c-445f-828e-98d197d6d0c6-kube-api-access-tdxpq\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299575 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-serving-cert\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299658 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299684 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-snapshots\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.299745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299709 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkkrq\" (UniqueName: \"kubernetes.io/projected/1b07bb5f-3052-47ea-afae-018bb50a9e43-kube-api-access-fkkrq\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299778 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299791 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299852 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:28.799832978 +0000 UTC m=+43.589920893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299852 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299889 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:28.799879606 +0000 UTC m=+43.589967518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299915 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:28.300388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.299981 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:28.79996456 +0000 UTC m=+43.590052497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:28.300749 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.300410 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.300749 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.300479 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-snapshots\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.300749 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.299780 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-tmp\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.301086 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.301060 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/816d33a9-20d0-440b-a9f3-914311397007-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.301236 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.301187 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7425c4b6-381c-445f-828e-98d197d6d0c6-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.301479 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.301457 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.303126 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.303107 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-serving-cert\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.308764 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.308539 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdxpq\" (UniqueName: \"kubernetes.io/projected/7425c4b6-381c-445f-828e-98d197d6d0c6-kube-api-access-tdxpq\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.309484 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.309456 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7lcg\" (UniqueName: \"kubernetes.io/projected/5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6-kube-api-access-p7lcg\") pod \"insights-operator-68d9dd94cf-7rwks\" (UID: \"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6\") " pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.310496 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.310468 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkkrq\" (UniqueName: \"kubernetes.io/projected/1b07bb5f-3052-47ea-afae-018bb50a9e43-kube-api-access-fkkrq\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.320353 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.320143 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" Jul 13 16:40:28.344650 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.344555 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:28.373063 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.370166 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" Jul 13 16:40:28.381243 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.381213 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl"] Jul 13 16:40:28.428573 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.428549 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f"] Jul 13 16:40:28.432525 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:28.432488 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1406b058_b79d_45d3_9a4d_2f0a3012f7ff.slice/crio-189eeceed1d798b3138b91db5a06f46f38b729ff355898e8edd9e2c9569dfdaf WatchSource:0}: Error finding container 189eeceed1d798b3138b91db5a06f46f38b729ff355898e8edd9e2c9569dfdaf: Status 404 returned error can't find the container with id 189eeceed1d798b3138b91db5a06f46f38b729ff355898e8edd9e2c9569dfdaf Jul 13 16:40:28.453219 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.453190 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" Jul 13 16:40:28.508600 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.508518 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf"] Jul 13 16:40:28.512985 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:28.512945 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67d6774c_170b_4704_8f96_3eab4de7fbf1.slice/crio-bd0524f49e85f65f2eb5b73dfd799630bea34e6be40167cd6d50788b5ad42e10 WatchSource:0}: Error finding container bd0524f49e85f65f2eb5b73dfd799630bea34e6be40167cd6d50788b5ad42e10: Status 404 returned error can't find the container with id bd0524f49e85f65f2eb5b73dfd799630bea34e6be40167cd6d50788b5ad42e10 Jul 13 16:40:28.544666 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.544593 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kmkjt"] Jul 13 16:40:28.549461 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:28.549431 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cb9f57_2e10_4550_b60f_6d7956a460ea.slice/crio-812c9e593ac003e0c48b93c9fc9b70d223f0b060b413ed5b6ce6870dd81f8da1 WatchSource:0}: Error finding container 812c9e593ac003e0c48b93c9fc9b70d223f0b060b413ed5b6ce6870dd81f8da1: Status 404 returned error can't find the container with id 812c9e593ac003e0c48b93c9fc9b70d223f0b060b413ed5b6ce6870dd81f8da1 Jul 13 16:40:28.562290 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.562266 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld"] Jul 13 16:40:28.564978 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:28.564951 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680fabb0_064a_4fc1_89f9_53d8c010bdd0.slice/crio-c91862265e84f7dac173584a713cbce90918253d1c06c84dc882ba16db29cf56 WatchSource:0}: Error finding container c91862265e84f7dac173584a713cbce90918253d1c06c84dc882ba16db29cf56: Status 404 returned error can't find the container with id c91862265e84f7dac173584a713cbce90918253d1c06c84dc882ba16db29cf56 Jul 13 16:40:28.602696 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.602670 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.602843 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.602779 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:28.602843 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.602814 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:28.602955 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.602876 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.602856239 +0000 UTC m=+44.392944165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:28.602955 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.602943 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.602925269 +0000 UTC m=+44.393013228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:28.620378 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.620337 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-7rwks"] Jul 13 16:40:28.622726 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:28.622705 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb4dc3e_ec0a_4ef1_b94f_5579b3d374b6.slice/crio-3bf2445c32678d2ab734d52867e134fc1cd1b4e340dd90592b1eb5db794bc603 WatchSource:0}: Error finding container 3bf2445c32678d2ab734d52867e134fc1cd1b4e340dd90592b1eb5db794bc603: Status 404 returned error can't find the container with id 3bf2445c32678d2ab734d52867e134fc1cd1b4e340dd90592b1eb5db794bc603 Jul 13 16:40:28.804710 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.804627 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:28.804843 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.804734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:28.804843 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804780 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:28.804843 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.804787 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:28.804975 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804842 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.804824905 +0000 UTC m=+44.594912837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:28.804975 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804859 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:28.804975 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804875 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:28.804975 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804929 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.804917974 +0000 UTC m=+44.595005885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:28.804975 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:28.804946 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:29.804939446 +0000 UTC m=+44.595027358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:28.948385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.948351 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kmkjt" event={"ID":"d5cb9f57-2e10-4550-b60f-6d7956a460ea","Type":"ContainerStarted","Data":"812c9e593ac003e0c48b93c9fc9b70d223f0b060b413ed5b6ce6870dd81f8da1"} Jul 13 16:40:28.949435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.949413 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" event={"ID":"1406b058-b79d-45d3-9a4d-2f0a3012f7ff","Type":"ContainerStarted","Data":"189eeceed1d798b3138b91db5a06f46f38b729ff355898e8edd9e2c9569dfdaf"} Jul 13 16:40:28.950400 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.950376 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" event={"ID":"f6a72c4e-2845-4db9-9cae-ee53a2a8daad","Type":"ContainerStarted","Data":"ac0622af8881bfdedaf2155f285cb695ab967fe4699f3ed6dc016f5816cfe6b0"} Jul 13 16:40:28.951396 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.951377 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" event={"ID":"680fabb0-064a-4fc1-89f9-53d8c010bdd0","Type":"ContainerStarted","Data":"c91862265e84f7dac173584a713cbce90918253d1c06c84dc882ba16db29cf56"} Jul 13 16:40:28.952399 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.952381 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" event={"ID":"67d6774c-170b-4704-8f96-3eab4de7fbf1","Type":"ContainerStarted","Data":"bd0524f49e85f65f2eb5b73dfd799630bea34e6be40167cd6d50788b5ad42e10"} Jul 13 16:40:28.953335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:28.953318 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" event={"ID":"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6","Type":"ContainerStarted","Data":"3bf2445c32678d2ab734d52867e134fc1cd1b4e340dd90592b1eb5db794bc603"} Jul 13 16:40:29.611721 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.611681 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:29.611887 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.611821 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:29.612012 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.611994 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.611975049 +0000 UTC m=+46.402062983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:29.612121 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.612033 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:29.612121 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.612093 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.612080806 +0000 UTC m=+46.402168732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:29.713299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.713261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:29.713474 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.713433 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:29.713474 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.713454 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:29.713576 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.713514 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:37.713494276 +0000 UTC m=+52.503582201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:29.814997 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.814967 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:29.815168 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.814855 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:29.815168 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815061 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.815023536 +0000 UTC m=+46.605111462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:29.815304 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.815167 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:29.815304 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.815217 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:29.815304 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815289 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:29.815453 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815306 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:29.815453 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815323 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:37.815311341 +0000 UTC m=+52.605399256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:29.815453 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815359 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.815343793 +0000 UTC m=+46.605431710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:29.815453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:29.815426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:29.815655 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815540 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:29.815655 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:29.815587 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.815573251 +0000 UTC m=+46.605661169 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:31.633939 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:31.633897 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:31.634358 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:31.634025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:31.634358 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.634074 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:31.634358 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.634145 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:35.634128618 +0000 UTC m=+50.424216551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:31.634358 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.634160 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:35.634154084 +0000 UTC m=+50.424241996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:31.836136 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:31.836093 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:31.836321 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836286 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:31.836388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:31.836327 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:31.836388 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836358 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:35.836340431 +0000 UTC m=+50.626428367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:31.836495 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836401 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:31.836495 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836436 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:35.836425873 +0000 UTC m=+50.626513801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:31.836495 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:31.836472 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:31.836599 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836544 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:31.836599 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:31.836576 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:35.836564561 +0000 UTC m=+50.626652477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:35.672268 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.672228 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:35.672745 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.672354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:35.672745 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.672496 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.672477882 +0000 UTC m=+58.462565800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:35.672745 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.672591 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:35.672745 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.672630 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.672619116 +0000 UTC m=+58.462707031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:35.875508 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.875470 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:35.875688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.875545 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:35.875688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.875625 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:35.875804 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.875775 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:35.875854 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.875840 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.87582155 +0000 UTC m=+58.665909485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:35.876067 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.876023 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:35.876262 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.876083 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:35.876262 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.876104 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.876089181 +0000 UTC m=+58.666177096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:35.876262 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:35.876149 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.876131178 +0000 UTC m=+58.666219096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:35.891024 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.890986 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lq2g9"] Jul 13 16:40:35.907923 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.907885 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lq2g9"] Jul 13 16:40:35.908130 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.908015 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.910778 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.910741 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 16:40:35.911243 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.911159 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 16:40:35.911481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.911449 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntdf5\"" Jul 13 16:40:35.976288 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.976182 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.976288 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.976277 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f1f6b35-4aea-4570-bf35-4f6e46680e83-data-volume\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.976524 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.976323 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqjvl\" (UniqueName: \"kubernetes.io/projected/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-api-access-wqjvl\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.976524 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.976352 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f1f6b35-4aea-4570-bf35-4f6e46680e83-crio-socket\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.976524 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.976401 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:35.977891 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.977836 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" event={"ID":"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6","Type":"ContainerStarted","Data":"0b2faedcc4abf8dd57e88815992e4e0c716ba9ac66b2d7a699d4779d3576ba30"} Jul 13 16:40:35.980120 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.980093 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/0.log" Jul 13 16:40:35.980230 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.980133 2569 generic.go:356] "Generic (PLEG): container finished" podID="d5cb9f57-2e10-4550-b60f-6d7956a460ea" containerID="e42a1901acdeb860c7e3ed6d2329ac637bff5a5161841f7045b8f97f4b37583d" exitCode=255 Jul 13 16:40:35.980230 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.980217 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kmkjt" event={"ID":"d5cb9f57-2e10-4550-b60f-6d7956a460ea","Type":"ContainerDied","Data":"e42a1901acdeb860c7e3ed6d2329ac637bff5a5161841f7045b8f97f4b37583d"} Jul 13 16:40:35.980755 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.980598 2569 scope.go:117] "RemoveContainer" containerID="e42a1901acdeb860c7e3ed6d2329ac637bff5a5161841f7045b8f97f4b37583d" Jul 13 16:40:35.982733 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.982710 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" event={"ID":"1406b058-b79d-45d3-9a4d-2f0a3012f7ff","Type":"ContainerStarted","Data":"d0890cb77230647f907435ea9d2ed55b534f723facf9711e64827c944d685f7f"} Jul 13 16:40:35.984747 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.984721 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" event={"ID":"f6a72c4e-2845-4db9-9cae-ee53a2a8daad","Type":"ContainerStarted","Data":"474cc99abdf0f816311eb340c63d7b7edc8e6d2c7db0c4855645c592d72907e7"} Jul 13 16:40:35.986608 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.986568 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" event={"ID":"680fabb0-064a-4fc1-89f9-53d8c010bdd0","Type":"ContainerStarted","Data":"4a682684afbe6707185b0129b01683670657372c92c4e5a66405c0c005933f40"} Jul 13 16:40:35.988002 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.987974 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" event={"ID":"67d6774c-170b-4704-8f96-3eab4de7fbf1","Type":"ContainerStarted","Data":"2a5ef0d18fc4c740baa61f28ab007eb12894f1f13bc952f7daed331ead962831"} Jul 13 16:40:35.997255 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:35.996284 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" podStartSLOduration=1.570341026 podStartE2EDuration="7.996266076s" totalImagesPullingTime="6.42592505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:35.995694254 +0000 UTC m=+50.785782188" watchObservedRunningTime="2026-07-13 16:40:35.996266076 +0000 UTC m=+50.786354005" Jul 13 16:40:36.013431 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.013387 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-4tqkf" podStartSLOduration=2.6928462250000003 podStartE2EDuration="9.013370144s" totalImagesPullingTime="6.320523919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:36.012334164 +0000 UTC m=+50.802422100" watchObservedRunningTime="2026-07-13 16:40:36.013370144 +0000 UTC m=+50.803458080" Jul 13 16:40:36.030925 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.030886 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-ksk5f" podStartSLOduration=2.405127677 podStartE2EDuration="9.03087175s" totalImagesPullingTime="6.625744073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:36.029689054 +0000 UTC m=+50.819776989" watchObservedRunningTime="2026-07-13 16:40:36.03087175 +0000 UTC m=+50.820959685" Jul 13 16:40:36.046265 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.046215 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" podStartSLOduration=2.56285459 podStartE2EDuration="9.046196179s" totalImagesPullingTime="6.483341589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:36.045370847 +0000 UTC m=+50.835458781" watchObservedRunningTime="2026-07-13 16:40:36.046196179 +0000 UTC m=+50.836284095" Jul 13 16:40:36.078212 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.078025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.078212 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.078184 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f1f6b35-4aea-4570-bf35-4f6e46680e83-data-volume\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.078488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.078306 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqjvl\" (UniqueName: \"kubernetes.io/projected/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-api-access-wqjvl\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.078691 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.078634 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f1f6b35-4aea-4570-bf35-4f6e46680e83-crio-socket\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.078793 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.078776 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.080074 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f1f6b35-4aea-4570-bf35-4f6e46680e83-crio-socket\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.080821 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.080876 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:36.580858215 +0000 UTC m=+51.370946140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : secret "insights-runtime-extractor-tls" not found Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.081116 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.081157 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:36.581142623 +0000 UTC m=+51.371230558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : configmap "kube-rbac-proxy" not found Jul 13 16:40:36.081882 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.081825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f1f6b35-4aea-4570-bf35-4f6e46680e83-data-volume\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.087194 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.086995 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" podStartSLOduration=2.434621667 podStartE2EDuration="9.08697886s" totalImagesPullingTime="6.652357193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:36.086056158 +0000 UTC m=+50.876144093" watchObservedRunningTime="2026-07-13 16:40:36.08697886 +0000 UTC m=+50.877066795" Jul 13 16:40:36.094192 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.094148 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqjvl\" (UniqueName: \"kubernetes.io/projected/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-api-access-wqjvl\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.583930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.583899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.584138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.584015 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:36.584138 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.584085 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 16:40:36.584138 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.584120 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 16:40:36.584292 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.584142 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:37.584125902 +0000 UTC m=+52.374213837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : secret "insights-runtime-extractor-tls" not found Jul 13 16:40:36.584292 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.584186 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:37.584168514 +0000 UTC m=+52.374256427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : configmap "kube-rbac-proxy" not found Jul 13 16:40:36.895526 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.895501 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2ppc_4cc7aca3-ab5d-4c42-af22-45450b82be31/dns-node-resolver/0.log" Jul 13 16:40:36.993428 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.993397 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:40:36.993831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.993815 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/0.log" Jul 13 16:40:36.993884 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.993853 2569 generic.go:356] "Generic (PLEG): container finished" podID="d5cb9f57-2e10-4550-b60f-6d7956a460ea" containerID="e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e" exitCode=255 Jul 13 16:40:36.993984 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.993956 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kmkjt" event={"ID":"d5cb9f57-2e10-4550-b60f-6d7956a460ea","Type":"ContainerDied","Data":"e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e"} Jul 13 16:40:36.994059 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.994026 2569 scope.go:117] "RemoveContainer" containerID="e42a1901acdeb860c7e3ed6d2329ac637bff5a5161841f7045b8f97f4b37583d" Jul 13 16:40:36.994196 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:36.994181 2569 scope.go:117] "RemoveContainer" containerID="e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e" Jul 13 16:40:36.994381 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:36.994358 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kmkjt_openshift-console-operator(d5cb9f57-2e10-4550-b60f-6d7956a460ea)\"" pod="openshift-console-operator/console-operator-7866497494-kmkjt" podUID="d5cb9f57-2e10-4550-b60f-6d7956a460ea" Jul 13 16:40:37.593574 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.593530 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:37.593782 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.593663 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:37.593782 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.593683 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 16:40:37.593782 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.593759 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:39.593743769 +0000 UTC m=+54.383831682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : secret "insights-runtime-extractor-tls" not found Jul 13 16:40:37.594221 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.594199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f1f6b35-4aea-4570-bf35-4f6e46680e83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:37.794950 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.794913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:37.795129 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.795079 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 16:40:37.795129 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.795097 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8686cfbcb8-wfvbz: secret "image-registry-tls" not found Jul 13 16:40:37.795214 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.795158 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls podName:07c6f0aa-b65f-46bd-9638-14af0a076ea0 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:53.795143987 +0000 UTC m=+68.585231899 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls") pod "image-registry-8686cfbcb8-wfvbz" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0") : secret "image-registry-tls" not found Jul 13 16:40:37.895541 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.895500 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:37.895891 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.895672 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 16:40:37.895891 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.895693 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7m9tz_c5b7a469-647c-4afe-91a5-e11edfb96827/node-ca/0.log" Jul 13 16:40:37.895891 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.895738 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls podName:267f22ff-d752-4298-923c-5baa9e0ad6e6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:53.895722974 +0000 UTC m=+68.685810886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls") pod "dns-default-zftr5" (UID: "267f22ff-d752-4298-923c-5baa9e0ad6e6") : secret "dns-default-metrics-tls" not found Jul 13 16:40:37.999055 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.999015 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:40:37.999367 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:37.999350 2569 scope.go:117] "RemoveContainer" containerID="e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e" Jul 13 16:40:37.999508 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:37.999490 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kmkjt_openshift-console-operator(d5cb9f57-2e10-4550-b60f-6d7956a460ea)\"" pod="openshift-console-operator/console-operator-7866497494-kmkjt" podUID="d5cb9f57-2e10-4550-b60f-6d7956a460ea" Jul 13 16:40:38.099671 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.099641 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd"] Jul 13 16:40:38.101822 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.101800 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" Jul 13 16:40:38.104452 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.104426 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qxgm8\"" Jul 13 16:40:38.104560 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.104426 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 16:40:38.104560 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.104485 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 16:40:38.111166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.111144 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd"] Jul 13 16:40:38.199126 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.199055 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8n7s\" (UniqueName: \"kubernetes.io/projected/3084f010-9683-4a3e-afdb-ab8edae69544-kube-api-access-r8n7s\") pod \"migrator-7f59b54f6c-rzqdd\" (UID: \"3084f010-9683-4a3e-afdb-ab8edae69544\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" Jul 13 16:40:38.299755 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.299722 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8n7s\" (UniqueName: \"kubernetes.io/projected/3084f010-9683-4a3e-afdb-ab8edae69544-kube-api-access-r8n7s\") pod \"migrator-7f59b54f6c-rzqdd\" (UID: \"3084f010-9683-4a3e-afdb-ab8edae69544\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" Jul 13 16:40:38.308167 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.308144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8n7s\" (UniqueName: \"kubernetes.io/projected/3084f010-9683-4a3e-afdb-ab8edae69544-kube-api-access-r8n7s\") pod \"migrator-7f59b54f6c-rzqdd\" (UID: \"3084f010-9683-4a3e-afdb-ab8edae69544\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" Jul 13 16:40:38.345597 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.345565 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:38.345704 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.345608 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:38.410899 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.410872 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" Jul 13 16:40:38.534746 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.534704 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd"] Jul 13 16:40:38.537295 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:38.537268 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3084f010_9683_4a3e_afdb_ab8edae69544.slice/crio-23cf5c981a017368c9d8c2a39bf434bbb9f85c9b984515ae62fbe790437528b6 WatchSource:0}: Error finding container 23cf5c981a017368c9d8c2a39bf434bbb9f85c9b984515ae62fbe790437528b6: Status 404 returned error can't find the container with id 23cf5c981a017368c9d8c2a39bf434bbb9f85c9b984515ae62fbe790437528b6 Jul 13 16:40:38.695634 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:38.695599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9b2j_25174820-93a8-4044-8154-889410fec806/serve-healthcheck-canary/0.log" Jul 13 16:40:39.004248 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:39.004211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" event={"ID":"3084f010-9683-4a3e-afdb-ab8edae69544","Type":"ContainerStarted","Data":"23cf5c981a017368c9d8c2a39bf434bbb9f85c9b984515ae62fbe790437528b6"} Jul 13 16:40:39.004605 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:39.004509 2569 scope.go:117] "RemoveContainer" containerID="e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e" Jul 13 16:40:39.004684 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:39.004664 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kmkjt_openshift-console-operator(d5cb9f57-2e10-4550-b60f-6d7956a460ea)\"" pod="openshift-console-operator/console-operator-7866497494-kmkjt" podUID="d5cb9f57-2e10-4550-b60f-6d7956a460ea" Jul 13 16:40:39.611366 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:39.611322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:39.611576 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:39.611502 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 16:40:39.611646 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:39.611580 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:43.611560477 +0000 UTC m=+58.401648391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : secret "insights-runtime-extractor-tls" not found Jul 13 16:40:40.009224 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:40.009198 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" event={"ID":"3084f010-9683-4a3e-afdb-ab8edae69544","Type":"ContainerStarted","Data":"ec7dfee9c46d70b042d33e71a07262bbf7d4912539e8b2f82622ff4103fdf36b"} Jul 13 16:40:41.014374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:41.014332 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" event={"ID":"3084f010-9683-4a3e-afdb-ab8edae69544","Type":"ContainerStarted","Data":"cb5c0104142c804eb4154a126a9eaec6fc812433d2cd1f087f27da1010b88579"} Jul 13 16:40:41.032456 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:41.032404 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-rzqdd" podStartSLOduration=1.6611740720000001 podStartE2EDuration="3.032386872s" totalImagesPullingTime="1.3712128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:41.031922404 +0000 UTC m=+55.822010341" watchObservedRunningTime="2026-07-13 16:40:41.032386872 +0000 UTC m=+55.822474803" Jul 13 16:40:43.645843 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.645807 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:43.646239 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.645944 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 16:40:43.646239 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.646015 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls podName:8f1f6b35-4aea-4570-bf35-4f6e46680e83 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:51.64599991 +0000 UTC m=+66.436087821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq2g9" (UID: "8f1f6b35-4aea-4570-bf35-4f6e46680e83") : secret "insights-runtime-extractor-tls" not found Jul 13 16:40:43.746547 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.746508 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:43.746707 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.746663 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:43.746707 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.746682 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 16:40:43.746786 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.746762 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:59.746745452 +0000 UTC m=+74.536833382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : secret "router-metrics-certs-default" not found Jul 13 16:40:43.746825 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.746795 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle podName:1d1f00f5-f46d-4962-8bfb-c49decad5994 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:59.746782686 +0000 UTC m=+74.536870598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle") pod "router-default-7657f6f484-wcq5g" (UID: "1d1f00f5-f46d-4962-8bfb-c49decad5994") : configmap references non-existent config key: service-ca.crt Jul 13 16:40:43.948494 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.948415 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:43.948494 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.948470 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:43.948668 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948567 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 16:40:43.948668 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:43.948600 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:43.948668 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948626 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls podName:1b07bb5f-3052-47ea-afae-018bb50a9e43 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:59.948611375 +0000 UTC m=+74.738699286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls") pod "cluster-samples-operator-65db858894-ktr9q" (UID: "1b07bb5f-3052-47ea-afae-018bb50a9e43") : secret "samples-operator-tls" not found Jul 13 16:40:43.948784 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948673 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 16:40:43.948784 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948684 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:43.948784 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948720 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert podName:816d33a9-20d0-440b-a9f3-914311397007 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:59.948708999 +0000 UTC m=+74.738796925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wjqdg" (UID: "816d33a9-20d0-440b-a9f3-914311397007") : secret "networking-console-plugin-cert" not found Jul 13 16:40:43.948784 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:43.948732 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls podName:7425c4b6-381c-445f-828e-98d197d6d0c6 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:59.94872606 +0000 UTC m=+74.738813971 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-fgszt" (UID: "7425c4b6-381c-445f-828e-98d197d6d0c6") : secret "cluster-monitoring-operator-tls" not found Jul 13 16:40:46.911753 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:46.911719 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7jxg" Jul 13 16:40:50.500006 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.499968 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:50.500513 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.500078 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:40:50.502998 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.502975 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 16:40:50.503093 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.503079 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 16:40:50.511089 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:50.511069 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 16:40:50.511204 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:40:50.511126 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs podName:a2326b3b-3b91-4d9f-957b-5a0cd8e78573 nodeName:}" failed. No retries permitted until 2026-07-13 16:41:54.511110215 +0000 UTC m=+129.301198127 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs") pod "network-metrics-daemon-4r5xc" (UID: "a2326b3b-3b91-4d9f-957b-5a0cd8e78573") : secret "metrics-daemon-secret" not found Jul 13 16:40:50.513177 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.513162 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81257490-18b6-48fa-8601-eef4639687e1-original-pull-secret\") pod \"global-pull-secret-syncer-grd9b\" (UID: \"81257490-18b6-48fa-8601-eef4639687e1\") " pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:50.600976 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.600940 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:50.603560 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.603543 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbxjq\" (UniqueName: \"kubernetes.io/projected/502b550d-c0f8-4cf7-961e-f8c9aa8fff9e-kube-api-access-dbxjq\") pod \"network-check-target-254w7\" (UID: \"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e\") " pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:50.737297 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.737257 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-grd9b" Jul 13 16:40:50.745786 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.745762 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-whvdd\"" Jul 13 16:40:50.753215 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.753155 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:50.864671 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.864642 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-grd9b"] Jul 13 16:40:50.867596 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:50.867572 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81257490_18b6_48fa_8601_eef4639687e1.slice/crio-ab3f0f068f363c0b7933a5701352607a21dfe84f36f18e824c38b1d01edd747a WatchSource:0}: Error finding container ab3f0f068f363c0b7933a5701352607a21dfe84f36f18e824c38b1d01edd747a: Status 404 returned error can't find the container with id ab3f0f068f363c0b7933a5701352607a21dfe84f36f18e824c38b1d01edd747a Jul 13 16:40:50.887183 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:50.887157 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-254w7"] Jul 13 16:40:50.889752 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:50.889721 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502b550d_c0f8_4cf7_961e_f8c9aa8fff9e.slice/crio-d1700ffa3b6d5cdddc2c52c16d19cf2c6d158a338710dd7618563aa9191284fc WatchSource:0}: Error finding container d1700ffa3b6d5cdddc2c52c16d19cf2c6d158a338710dd7618563aa9191284fc: Status 404 returned error can't find the container with id d1700ffa3b6d5cdddc2c52c16d19cf2c6d158a338710dd7618563aa9191284fc Jul 13 16:40:51.048247 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.048162 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-grd9b" event={"ID":"81257490-18b6-48fa-8601-eef4639687e1","Type":"ContainerStarted","Data":"ab3f0f068f363c0b7933a5701352607a21dfe84f36f18e824c38b1d01edd747a"} Jul 13 16:40:51.049664 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.049638 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-254w7" event={"ID":"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e","Type":"ContainerStarted","Data":"f61ad6d4129d435f1722ed4e33dd848e805c084b89ed2b69cb30e0f0c19f0d70"} Jul 13 16:40:51.049785 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.049677 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-254w7" event={"ID":"502b550d-c0f8-4cf7-961e-f8c9aa8fff9e","Type":"ContainerStarted","Data":"d1700ffa3b6d5cdddc2c52c16d19cf2c6d158a338710dd7618563aa9191284fc"} Jul 13 16:40:51.049785 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.049770 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:40:51.069074 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.069014 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-254w7" podStartSLOduration=65.069000746 podStartE2EDuration="1m5.069000746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:51.067010156 +0000 UTC m=+65.857098089" watchObservedRunningTime="2026-07-13 16:40:51.069000746 +0000 UTC m=+65.859088680" Jul 13 16:40:51.712382 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.712344 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:51.714975 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.714944 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f1f6b35-4aea-4570-bf35-4f6e46680e83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq2g9\" (UID: \"8f1f6b35-4aea-4570-bf35-4f6e46680e83\") " pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:51.824161 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.824126 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntdf5\"" Jul 13 16:40:51.831831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.831806 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lq2g9" Jul 13 16:40:51.969243 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:51.969163 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lq2g9"] Jul 13 16:40:51.972345 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:51.972314 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1f6b35_4aea_4570_bf35_4f6e46680e83.slice/crio-c20157dd2680a1632a59457e1007a84f69c02e71130c5078b7e7ea96880aa7c9 WatchSource:0}: Error finding container c20157dd2680a1632a59457e1007a84f69c02e71130c5078b7e7ea96880aa7c9: Status 404 returned error can't find the container with id c20157dd2680a1632a59457e1007a84f69c02e71130c5078b7e7ea96880aa7c9 Jul 13 16:40:52.054255 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:52.054212 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq2g9" event={"ID":"8f1f6b35-4aea-4570-bf35-4f6e46680e83","Type":"ContainerStarted","Data":"c20157dd2680a1632a59457e1007a84f69c02e71130c5078b7e7ea96880aa7c9"} Jul 13 16:40:52.714883 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:52.714841 2569 scope.go:117] "RemoveContainer" containerID="e9a80f4857d149324a2df954a1977896ab36d4a49f3cdd2f9057537d2241b09e" Jul 13 16:40:53.063526 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.063452 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:40:53.063697 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.063549 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kmkjt" event={"ID":"d5cb9f57-2e10-4550-b60f-6d7956a460ea","Type":"ContainerStarted","Data":"b218a3cd7b7932823076f83c9e35fe1deda92caa1644c13e3ce31b1e8cdb54b5"} Jul 13 16:40:53.064400 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.064375 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:53.611343 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.611307 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-kmkjt" Jul 13 16:40:53.630645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.629863 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-kmkjt" podStartSLOduration=20.132048891 podStartE2EDuration="26.62984441s" totalImagesPullingTime="6.497795519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:53.086353634 +0000 UTC m=+67.876441568" watchObservedRunningTime="2026-07-13 16:40:53.62984441 +0000 UTC m=+68.419932344" Jul 13 16:40:53.833579 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.833509 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:53.836923 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.836896 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"image-registry-8686cfbcb8-wfvbz\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:53.849205 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.849176 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6ng2j\"" Jul 13 16:40:53.855987 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.855963 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:53.934966 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.934932 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:53.937667 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:53.937645 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/267f22ff-d752-4298-923c-5baa9e0ad6e6-metrics-tls\") pod \"dns-default-zftr5\" (UID: \"267f22ff-d752-4298-923c-5baa9e0ad6e6\") " pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:54.157797 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:54.157767 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mgdqr\"" Jul 13 16:40:54.165905 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:54.165880 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zftr5" Jul 13 16:40:58.312487 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:58.312457 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zftr5"] Jul 13 16:40:58.315573 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:58.315539 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267f22ff_d752_4298_923c_5baa9e0ad6e6.slice/crio-f15c872f02d75bd3cb37447c35c991b9c53a55296a4076392d5a73167ce97715 WatchSource:0}: Error finding container f15c872f02d75bd3cb37447c35c991b9c53a55296a4076392d5a73167ce97715: Status 404 returned error can't find the container with id f15c872f02d75bd3cb37447c35c991b9c53a55296a4076392d5a73167ce97715 Jul 13 16:40:58.335308 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:58.335282 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:40:58.337906 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:40:58.337881 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c6f0aa_b65f_46bd_9638_14af0a076ea0.slice/crio-8dd56b3b0d37be79d79f01c60aa32c899dbd05bf973fe78f5828353654d60240 WatchSource:0}: Error finding container 8dd56b3b0d37be79d79f01c60aa32c899dbd05bf973fe78f5828353654d60240: Status 404 returned error can't find the container with id 8dd56b3b0d37be79d79f01c60aa32c899dbd05bf973fe78f5828353654d60240 Jul 13 16:40:59.085617 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.085576 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" event={"ID":"07c6f0aa-b65f-46bd-9638-14af0a076ea0","Type":"ContainerStarted","Data":"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77"} Jul 13 16:40:59.085802 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.085632 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" event={"ID":"07c6f0aa-b65f-46bd-9638-14af0a076ea0","Type":"ContainerStarted","Data":"8dd56b3b0d37be79d79f01c60aa32c899dbd05bf973fe78f5828353654d60240"} Jul 13 16:40:59.085802 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.085680 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:40:59.086758 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.086732 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zftr5" event={"ID":"267f22ff-d752-4298-923c-5baa9e0ad6e6","Type":"ContainerStarted","Data":"f15c872f02d75bd3cb37447c35c991b9c53a55296a4076392d5a73167ce97715"} Jul 13 16:40:59.087979 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.087955 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-grd9b" event={"ID":"81257490-18b6-48fa-8601-eef4639687e1","Type":"ContainerStarted","Data":"f4ab8105b06f7d2fed0e6ef48f7664b77d472a09c835d0feb76a8bb2dbab3d65"} Jul 13 16:40:59.089256 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.089235 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq2g9" event={"ID":"8f1f6b35-4aea-4570-bf35-4f6e46680e83","Type":"ContainerStarted","Data":"bb4c699cb3c7e4e0da5307f1b94ca17f0b3df54b272c54bd9e70928702d6da8d"} Jul 13 16:40:59.107368 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.107330 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" podStartSLOduration=73.107318867 podStartE2EDuration="1m13.107318867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:59.106165084 +0000 UTC m=+73.896253029" watchObservedRunningTime="2026-07-13 16:40:59.107318867 +0000 UTC m=+73.897406779" Jul 13 16:40:59.121493 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.121458 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-grd9b" podStartSLOduration=65.830734351 podStartE2EDuration="1m13.121444866s" totalImagesPullingTime="7.290710515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:59.120497854 +0000 UTC m=+73.910585792" watchObservedRunningTime="2026-07-13 16:40:59.121444866 +0000 UTC m=+73.911532799" Jul 13 16:40:59.784302 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.784268 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:59.784664 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.784327 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:59.784853 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.784834 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d1f00f5-f46d-4962-8bfb-c49decad5994-service-ca-bundle\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:59.786643 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.786624 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d1f00f5-f46d-4962-8bfb-c49decad5994-metrics-certs\") pod \"router-default-7657f6f484-wcq5g\" (UID: \"1d1f00f5-f46d-4962-8bfb-c49decad5994\") " pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:40:59.985991 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.985951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:59.986179 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.986027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:40:59.986179 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.986095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:59.988852 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.988819 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/816d33a9-20d0-440b-a9f3-914311397007-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wjqdg\" (UID: \"816d33a9-20d0-440b-a9f3-914311397007\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:40:59.988982 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.988893 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7425c4b6-381c-445f-828e-98d197d6d0c6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-fgszt\" (UID: \"7425c4b6-381c-445f-828e-98d197d6d0c6\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:40:59.988982 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:40:59.988935 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b07bb5f-3052-47ea-afae-018bb50a9e43-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ktr9q\" (UID: \"1b07bb5f-3052-47ea-afae-018bb50a9e43\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:41:00.068308 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.068236 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5s6dv\"" Jul 13 16:41:00.076437 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.076406 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:41:00.186658 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.186421 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j2nxs\"" Jul 13 16:41:00.193958 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.193935 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" Jul 13 16:41:00.216414 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.216382 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lc7lk\"" Jul 13 16:41:00.223540 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.223514 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" Jul 13 16:41:00.230642 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:00.230603 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1f00f5_f46d_4962_8bfb_c49decad5994.slice/crio-aa8daf571283196a1b9121b03a11c431eb0215aaf39cddf0175613fbb71717bb WatchSource:0}: Error finding container aa8daf571283196a1b9121b03a11c431eb0215aaf39cddf0175613fbb71717bb: Status 404 returned error can't find the container with id aa8daf571283196a1b9121b03a11c431eb0215aaf39cddf0175613fbb71717bb Jul 13 16:41:00.230642 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.230619 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7657f6f484-wcq5g"] Jul 13 16:41:00.241412 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.241391 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-655cv\"" Jul 13 16:41:00.249979 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.249932 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" Jul 13 16:41:00.373164 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.373084 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q"] Jul 13 16:41:00.404697 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.404666 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt"] Jul 13 16:41:00.408980 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:00.408951 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7425c4b6_381c_445f_828e_98d197d6d0c6.slice/crio-89302cabcbf999a8627380844eda897d9c8dc416d094c2cfe85e8176980ebd69 WatchSource:0}: Error finding container 89302cabcbf999a8627380844eda897d9c8dc416d094c2cfe85e8176980ebd69: Status 404 returned error can't find the container with id 89302cabcbf999a8627380844eda897d9c8dc416d094c2cfe85e8176980ebd69 Jul 13 16:41:00.431374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:00.431245 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wjqdg"] Jul 13 16:41:00.434776 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:00.434751 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816d33a9_20d0_440b_a9f3_914311397007.slice/crio-58342a8c0f9e646969ef8c4baa3270dec9c0a486774ff25e864151ce414344bf WatchSource:0}: Error finding container 58342a8c0f9e646969ef8c4baa3270dec9c0a486774ff25e864151ce414344bf: Status 404 returned error can't find the container with id 58342a8c0f9e646969ef8c4baa3270dec9c0a486774ff25e864151ce414344bf Jul 13 16:41:01.097828 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.097794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" event={"ID":"816d33a9-20d0-440b-a9f3-914311397007","Type":"ContainerStarted","Data":"58342a8c0f9e646969ef8c4baa3270dec9c0a486774ff25e864151ce414344bf"} Jul 13 16:41:01.098847 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.098825 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" event={"ID":"7425c4b6-381c-445f-828e-98d197d6d0c6","Type":"ContainerStarted","Data":"89302cabcbf999a8627380844eda897d9c8dc416d094c2cfe85e8176980ebd69"} Jul 13 16:41:01.099854 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.099829 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" event={"ID":"1b07bb5f-3052-47ea-afae-018bb50a9e43","Type":"ContainerStarted","Data":"218134cf88c0ebdd46306ac9e6204cbd0863627c485897357d7f9f74755dceba"} Jul 13 16:41:01.101271 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.101247 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7657f6f484-wcq5g" event={"ID":"1d1f00f5-f46d-4962-8bfb-c49decad5994","Type":"ContainerStarted","Data":"2284373c0fac14a72ee22cb16b0eccb4f4e73d151c599fbdca97b8197bd808d0"} Jul 13 16:41:01.101366 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.101288 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7657f6f484-wcq5g" event={"ID":"1d1f00f5-f46d-4962-8bfb-c49decad5994","Type":"ContainerStarted","Data":"aa8daf571283196a1b9121b03a11c431eb0215aaf39cddf0175613fbb71717bb"} Jul 13 16:41:01.123102 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:01.123055 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7657f6f484-wcq5g" podStartSLOduration=34.123026315 podStartE2EDuration="34.123026315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:01.121984836 +0000 UTC m=+75.912072772" watchObservedRunningTime="2026-07-13 16:41:01.123026315 +0000 UTC m=+75.913114248" Jul 13 16:41:02.077318 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.077271 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:41:02.080273 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.080248 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:41:02.083236 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.083212 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-bgv2v"] Jul 13 16:41:02.085900 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.085875 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:02.086057 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.086020 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:02.088162 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.088137 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.089899 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.089810 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 16:41:02.089899 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.089830 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nrw5g\"" Jul 13 16:41:02.091488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.090955 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 16:41:02.091488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.091322 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 16:41:02.091488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.091352 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 16:41:02.092286 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.092267 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8bczz\"" Jul 13 16:41:02.092388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.092346 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 16:41:02.092544 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.092526 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 16:41:02.092626 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.092574 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 16:41:02.102718 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.102692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.102734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.102796 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxj4h\" (UniqueName: \"kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.102896 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.102954 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.103007 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7nlf\" (UniqueName: \"kubernetes.io/projected/1cb7c776-b672-4346-8cb7-f88c8fb2a34e-kube-api-access-f7nlf\") pod \"downloads-68c9cbb497-bgv2v\" (UID: \"1cb7c776-b672-4346-8cb7-f88c8fb2a34e\") " pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:02.103171 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.103031 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.108016 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.107989 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq2g9" event={"ID":"8f1f6b35-4aea-4570-bf35-4f6e46680e83","Type":"ContainerStarted","Data":"965a1a7beb6208099249ff04b89e5df716371335374456f002197ad6a4b8cd56"} Jul 13 16:41:02.110375 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.110348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zftr5" event={"ID":"267f22ff-d752-4298-923c-5baa9e0ad6e6","Type":"ContainerStarted","Data":"7a645af9bcf236547fb678cb5f609c69c0b9c9c6536f58c9f3b991540ffb8828"} Jul 13 16:41:02.110495 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.110408 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zftr5" event={"ID":"267f22ff-d752-4298-923c-5baa9e0ad6e6","Type":"ContainerStarted","Data":"b9e6484c49e019c9736619c4977c3e03e93eb8cf03c2baf2c7aae1f623c20624"} Jul 13 16:41:02.110495 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.110454 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zftr5" Jul 13 16:41:02.112088 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.112061 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" event={"ID":"816d33a9-20d0-440b-a9f3-914311397007","Type":"ContainerStarted","Data":"e70a0c512f38b3a7b2720ace6d9c2bd69ea8075d1b87ad2f445763b0fd8c6cc9"} Jul 13 16:41:02.112375 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.112352 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:41:02.113678 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.113657 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7657f6f484-wcq5g" Jul 13 16:41:02.116385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.116335 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-bgv2v"] Jul 13 16:41:02.117681 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.117660 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:02.183464 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.183363 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zftr5" podStartSLOduration=38.387883227 podStartE2EDuration="41.183343068s" totalImagesPullingTime="2.795459841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:02.181962626 +0000 UTC m=+76.972050561" watchObservedRunningTime="2026-07-13 16:41:02.183343068 +0000 UTC m=+76.973431002" Jul 13 16:41:02.204349 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.203646 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204349 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.203776 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204349 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.203863 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7nlf\" (UniqueName: \"kubernetes.io/projected/1cb7c776-b672-4346-8cb7-f88c8fb2a34e-kube-api-access-f7nlf\") pod \"downloads-68c9cbb497-bgv2v\" (UID: \"1cb7c776-b672-4346-8cb7-f88c8fb2a34e\") " pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:02.204349 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.203896 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.204487 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.204530 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204688 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.204559 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxj4h\" (UniqueName: \"kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.204968 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.204941 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.205453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.205428 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.205680 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.205450 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.207363 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.207323 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.208491 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.208451 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.222709 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.222654 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7nlf\" (UniqueName: \"kubernetes.io/projected/1cb7c776-b672-4346-8cb7-f88c8fb2a34e-kube-api-access-f7nlf\") pod \"downloads-68c9cbb497-bgv2v\" (UID: \"1cb7c776-b672-4346-8cb7-f88c8fb2a34e\") " pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:02.226270 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.226218 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-wjqdg" podStartSLOduration=32.750146627 podStartE2EDuration="34.22619848s" totalImagesPullingTime="1.476051853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:02.197945195 +0000 UTC m=+76.988033130" watchObservedRunningTime="2026-07-13 16:41:02.22619848 +0000 UTC m=+77.016286415" Jul 13 16:41:02.227554 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.227533 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxj4h\" (UniqueName: \"kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h\") pod \"console-7d864b8c99-9m8sm\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.398352 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.398316 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:02.405580 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.405527 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:02.574137 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.574005 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-bgv2v"] Jul 13 16:41:02.579715 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:02.579680 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb7c776_b672_4346_8cb7_f88c8fb2a34e.slice/crio-02b784b410177977c77ac85c5e0e825c64e70980bc0f5c08c1af3f8e885bd036 WatchSource:0}: Error finding container 02b784b410177977c77ac85c5e0e825c64e70980bc0f5c08c1af3f8e885bd036: Status 404 returned error can't find the container with id 02b784b410177977c77ac85c5e0e825c64e70980bc0f5c08c1af3f8e885bd036 Jul 13 16:41:02.597765 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:02.597711 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:02.602956 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:02.602893 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6697f218_00b3_46af_842f_9e0faf6395e4.slice/crio-6514710ca7ec46daa0599c2dee467629721d606b02e470da1ae74cda9adc9b38 WatchSource:0}: Error finding container 6514710ca7ec46daa0599c2dee467629721d606b02e470da1ae74cda9adc9b38: Status 404 returned error can't find the container with id 6514710ca7ec46daa0599c2dee467629721d606b02e470da1ae74cda9adc9b38 Jul 13 16:41:03.124117 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:03.124080 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-bgv2v" event={"ID":"1cb7c776-b672-4346-8cb7-f88c8fb2a34e","Type":"ContainerStarted","Data":"02b784b410177977c77ac85c5e0e825c64e70980bc0f5c08c1af3f8e885bd036"} Jul 13 16:41:03.125842 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:03.125780 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d864b8c99-9m8sm" event={"ID":"6697f218-00b3-46af-842f-9e0faf6395e4","Type":"ContainerStarted","Data":"6514710ca7ec46daa0599c2dee467629721d606b02e470da1ae74cda9adc9b38"} Jul 13 16:41:08.149464 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.149424 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d864b8c99-9m8sm" event={"ID":"6697f218-00b3-46af-842f-9e0faf6395e4","Type":"ContainerStarted","Data":"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957"} Jul 13 16:41:08.151105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.151073 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" event={"ID":"7425c4b6-381c-445f-828e-98d197d6d0c6","Type":"ContainerStarted","Data":"5b915fba8279303f3950ce58cf208c4b04e0844ce666d16485b4554df87b2de5"} Jul 13 16:41:08.153125 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.153084 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" event={"ID":"1b07bb5f-3052-47ea-afae-018bb50a9e43","Type":"ContainerStarted","Data":"bd7521dfeee87953b79a63cd9aca67feb9be5d2325e74cd658a79a8d79c51213"} Jul 13 16:41:08.153229 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.153143 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" event={"ID":"1b07bb5f-3052-47ea-afae-018bb50a9e43","Type":"ContainerStarted","Data":"f6944ec1c9da29a2c568503385fafdb4fadec87eebaef8b567fd2b9f6f780244"} Jul 13 16:41:08.155356 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.155329 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq2g9" event={"ID":"8f1f6b35-4aea-4570-bf35-4f6e46680e83","Type":"ContainerStarted","Data":"e7308602aa22d5d9568ac220e1b6575e1475f6832a763d68d854a34a49326580"} Jul 13 16:41:08.169895 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.169841 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d864b8c99-9m8sm" podStartSLOduration=1.251868049 podStartE2EDuration="6.169822884s" totalImagesPullingTime="4.917954835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:08.16871116 +0000 UTC m=+82.958799094" watchObservedRunningTime="2026-07-13 16:41:08.169822884 +0000 UTC m=+82.959910819" Jul 13 16:41:08.191944 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.191905 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lq2g9" podStartSLOduration=19.476324949 podStartE2EDuration="33.191893029s" totalImagesPullingTime="13.71556808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:08.190106316 +0000 UTC m=+82.980194250" watchObservedRunningTime="2026-07-13 16:41:08.191893029 +0000 UTC m=+82.981980963" Jul 13 16:41:08.207464 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.207418 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ktr9q" podStartSLOduration=35.752380085 podStartE2EDuration="41.207405881s" totalImagesPullingTime="5.455025796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:08.206760513 +0000 UTC m=+82.996848458" watchObservedRunningTime="2026-07-13 16:41:08.207405881 +0000 UTC m=+82.997493814" Jul 13 16:41:08.226029 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:08.225983 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-fgszt" podStartSLOduration=35.749090826 podStartE2EDuration="41.225970433s" totalImagesPullingTime="5.476879607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:08.225202365 +0000 UTC m=+83.015290299" watchObservedRunningTime="2026-07-13 16:41:08.225970433 +0000 UTC m=+83.016058368" Jul 13 16:41:12.128105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:12.128070 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zftr5" Jul 13 16:41:12.406293 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:12.406257 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:12.406471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:12.406329 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:12.407862 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:12.407835 2569 patch_prober.go:28] interesting pod/console-7d864b8c99-9m8sm container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" start-of-body= Jul 13 16:41:12.407997 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:12.407896 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d864b8c99-9m8sm" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" containerName="console" probeResult="failure" output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" Jul 13 16:41:13.862195 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:13.862162 2569 patch_prober.go:28] interesting pod/image-registry-8686cfbcb8-wfvbz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 16:41:13.862599 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:13.862229 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 16:41:19.358926 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:19.358887 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:20.099717 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:20.099682 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:41:21.213264 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.213225 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-bgv2v" event={"ID":"1cb7c776-b672-4346-8cb7-f88c8fb2a34e","Type":"ContainerStarted","Data":"7877dac92294aa566cc5b6cba306f1b50a7ad63abef79674d6c353ee1f966185"} Jul 13 16:41:21.213765 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.213493 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:21.227755 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.227726 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-bgv2v" Jul 13 16:41:21.231615 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.231587 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-529j7"] Jul 13 16:41:21.238689 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.238603 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.241178 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.241152 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bgr75\"" Jul 13 16:41:21.241314 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.241286 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 16:41:21.241387 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.241373 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 16:41:21.241445 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.241391 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 16:41:21.242181 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.242156 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 16:41:21.249501 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.249454 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-bgv2v" podStartSLOduration=0.949466477 podStartE2EDuration="19.249437093s" totalImagesPullingTime="18.299970616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:21.248633034 +0000 UTC m=+96.038720969" watchObservedRunningTime="2026-07-13 16:41:21.249437093 +0000 UTC m=+96.039525028" Jul 13 16:41:21.382450 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382402 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-sys\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382450 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382444 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-root\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382697 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382566 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382697 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382631 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-metrics-client-ca\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382697 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382691 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-textfile\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382850 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382729 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382850 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382774 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-wtmp\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382950 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382852 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmvk\" (UniqueName: \"kubernetes.io/projected/b4b54637-6577-497b-9330-c3be6a3e37a1-kube-api-access-rrmvk\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.382950 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.382892 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-tls\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.483938 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.483846 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmvk\" (UniqueName: \"kubernetes.io/projected/b4b54637-6577-497b-9330-c3be6a3e37a1-kube-api-access-rrmvk\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.483938 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.483896 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-tls\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.483938 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.483923 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-sys\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484238 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.483943 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-root\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484238 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.483994 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484238 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484024 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-metrics-client-ca\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484238 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484088 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-sys\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484454 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484105 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-textfile\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484516 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484447 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-textfile\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484571 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484628 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484578 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-wtmp\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484756 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484734 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-metrics-client-ca\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.484756 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.484747 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-wtmp\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.485158 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.485132 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4b54637-6577-497b-9330-c3be6a3e37a1-root\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.485276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.485246 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.486824 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.486779 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-tls\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.487465 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.487441 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4b54637-6577-497b-9330-c3be6a3e37a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.492482 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.492458 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmvk\" (UniqueName: \"kubernetes.io/projected/b4b54637-6577-497b-9330-c3be6a3e37a1-kube-api-access-rrmvk\") pod \"node-exporter-529j7\" (UID: \"b4b54637-6577-497b-9330-c3be6a3e37a1\") " pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.551587 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:21.551545 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-529j7" Jul 13 16:41:21.564820 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:21.564781 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b54637_6577_497b_9330_c3be6a3e37a1.slice/crio-5799de2204c4edcde60659bb0f87328ae4f0103c2fee2029f3223631b5dc8622 WatchSource:0}: Error finding container 5799de2204c4edcde60659bb0f87328ae4f0103c2fee2029f3223631b5dc8622: Status 404 returned error can't find the container with id 5799de2204c4edcde60659bb0f87328ae4f0103c2fee2029f3223631b5dc8622 Jul 13 16:41:22.058063 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:22.058002 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-254w7" Jul 13 16:41:22.221819 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:22.221362 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-529j7" event={"ID":"b4b54637-6577-497b-9330-c3be6a3e37a1","Type":"ContainerStarted","Data":"5799de2204c4edcde60659bb0f87328ae4f0103c2fee2029f3223631b5dc8622"} Jul 13 16:41:23.229838 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:23.229804 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-529j7" event={"ID":"b4b54637-6577-497b-9330-c3be6a3e37a1","Type":"ContainerStarted","Data":"7df07d285f9fd0788125edb79a32cb7cc14ea1ac34cf0527d6463310bc4f2c75"} Jul 13 16:41:24.236144 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:24.236109 2569 generic.go:356] "Generic (PLEG): container finished" podID="b4b54637-6577-497b-9330-c3be6a3e37a1" containerID="7df07d285f9fd0788125edb79a32cb7cc14ea1ac34cf0527d6463310bc4f2c75" exitCode=0 Jul 13 16:41:24.236709 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:24.236180 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-529j7" event={"ID":"b4b54637-6577-497b-9330-c3be6a3e37a1","Type":"ContainerDied","Data":"7df07d285f9fd0788125edb79a32cb7cc14ea1ac34cf0527d6463310bc4f2c75"} Jul 13 16:41:24.455542 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:24.455511 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:41:25.243276 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:25.243238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-529j7" event={"ID":"b4b54637-6577-497b-9330-c3be6a3e37a1","Type":"ContainerStarted","Data":"3615253ac9205034fd4f6cbfc2ecce56ab33ce4ffb4620ea20cda16cb6116ca0"} Jul 13 16:41:25.243739 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:25.243293 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-529j7" event={"ID":"b4b54637-6577-497b-9330-c3be6a3e37a1","Type":"ContainerStarted","Data":"183c9a24af0702e2ccc2c2b1e84ca1f2ce7d65db61fa725f3dfe8db384f8e9d1"} Jul 13 16:41:25.265275 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:25.265219 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-529j7" podStartSLOduration=2.619994298 podStartE2EDuration="4.265196729s" totalImagesPullingTime="1.546238287s" totalInitContainerRuntime="98.964144ms" isStatefulPod=true podCreationTimestamp="2026-07-13 16:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:25.262836417 +0000 UTC m=+100.052924353" watchObservedRunningTime="2026-07-13 16:41:25.265196729 +0000 UTC m=+100.055284666" Jul 13 16:41:44.383648 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.383589 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d864b8c99-9m8sm" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" containerName="console" containerID="cri-o://f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957" gracePeriod=15 Jul 13 16:41:44.667341 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.667315 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:44.685088 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685059 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685101 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685173 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685220 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685407 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685250 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685407 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685280 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxj4h\" (UniqueName: \"kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h\") pod \"6697f218-00b3-46af-842f-9e0faf6395e4\" (UID: \"6697f218-00b3-46af-842f-9e0faf6395e4\") " Jul 13 16:41:44.685785 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685415 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca" (OuterVolumeSpecName: "service-ca") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:41:44.685846 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685803 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:44.685900 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685850 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:41:44.685900 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.685880 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config" (OuterVolumeSpecName: "console-config") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:41:44.687977 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.687934 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h" (OuterVolumeSpecName: "kube-api-access-fxj4h") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "kube-api-access-fxj4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:41:44.688102 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.688002 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:41:44.688102 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.688026 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6697f218-00b3-46af-842f-9e0faf6395e4" (UID: "6697f218-00b3-46af-842f-9e0faf6395e4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:41:44.786239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.786209 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-console-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:44.786239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.786236 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:44.786239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.786246 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6697f218-00b3-46af-842f-9e0faf6395e4-console-oauth-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:44.786535 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.786255 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6697f218-00b3-46af-842f-9e0faf6395e4-oauth-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:44.786535 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:44.786264 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxj4h\" (UniqueName: \"kubernetes.io/projected/6697f218-00b3-46af-842f-9e0faf6395e4-kube-api-access-fxj4h\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:45.325483 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.325450 2569 generic.go:356] "Generic (PLEG): container finished" podID="6697f218-00b3-46af-842f-9e0faf6395e4" containerID="f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957" exitCode=0 Jul 13 16:41:45.325657 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.325504 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d864b8c99-9m8sm" Jul 13 16:41:45.325657 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.325539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d864b8c99-9m8sm" event={"ID":"6697f218-00b3-46af-842f-9e0faf6395e4","Type":"ContainerDied","Data":"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957"} Jul 13 16:41:45.325657 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.325622 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d864b8c99-9m8sm" event={"ID":"6697f218-00b3-46af-842f-9e0faf6395e4","Type":"ContainerDied","Data":"6514710ca7ec46daa0599c2dee467629721d606b02e470da1ae74cda9adc9b38"} Jul 13 16:41:45.325657 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.325643 2569 scope.go:117] "RemoveContainer" containerID="f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957" Jul 13 16:41:45.334048 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.334019 2569 scope.go:117] "RemoveContainer" containerID="f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957" Jul 13 16:41:45.334310 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:41:45.334290 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957\": container with ID starting with f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957 not found: ID does not exist" containerID="f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957" Jul 13 16:41:45.334357 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.334318 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957"} err="failed to get container status \"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957\": rpc error: code = NotFound desc = could not find container \"f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957\": container with ID starting with f15a93e68ee16ccdcb686022b5f75527efa9b3d7c1c0e94dc16672ac996e3957 not found: ID does not exist" Jul 13 16:41:45.344902 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.344881 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:45.348655 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.348633 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d864b8c99-9m8sm"] Jul 13 16:41:45.719405 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:45.719371 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" path="/var/lib/kubelet/pods/6697f218-00b3-46af-842f-9e0faf6395e4/volumes" Jul 13 16:41:46.330245 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.330154 2569 generic.go:356] "Generic (PLEG): container finished" podID="f6a72c4e-2845-4db9-9cae-ee53a2a8daad" containerID="474cc99abdf0f816311eb340c63d7b7edc8e6d2c7db0c4855645c592d72907e7" exitCode=0 Jul 13 16:41:46.330383 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.330233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" event={"ID":"f6a72c4e-2845-4db9-9cae-ee53a2a8daad","Type":"ContainerDied","Data":"474cc99abdf0f816311eb340c63d7b7edc8e6d2c7db0c4855645c592d72907e7"} Jul 13 16:41:46.330645 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.330622 2569 scope.go:117] "RemoveContainer" containerID="474cc99abdf0f816311eb340c63d7b7edc8e6d2c7db0c4855645c592d72907e7" Jul 13 16:41:46.331699 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.331675 2569 generic.go:356] "Generic (PLEG): container finished" podID="5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6" containerID="0b2faedcc4abf8dd57e88815992e4e0c716ba9ac66b2d7a699d4779d3576ba30" exitCode=0 Jul 13 16:41:46.331784 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.331749 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" event={"ID":"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6","Type":"ContainerDied","Data":"0b2faedcc4abf8dd57e88815992e4e0c716ba9ac66b2d7a699d4779d3576ba30"} Jul 13 16:41:46.332028 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:46.332015 2569 scope.go:117] "RemoveContainer" containerID="0b2faedcc4abf8dd57e88815992e4e0c716ba9ac66b2d7a699d4779d3576ba30" Jul 13 16:41:47.236871 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:47.236812 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7657f6f484-wcq5g_1d1f00f5-f46d-4962-8bfb-c49decad5994/router/0.log" Jul 13 16:41:47.254663 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:47.254628 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9b2j_25174820-93a8-4044-8154-889410fec806/serve-healthcheck-canary/0.log" Jul 13 16:41:47.337114 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:47.337083 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-2n6wl" event={"ID":"f6a72c4e-2845-4db9-9cae-ee53a2a8daad","Type":"ContainerStarted","Data":"29adf9bdb3f454a75ef2d18448f918d2fc8c5aa25e1ce1c3266281cae5c29c3b"} Jul 13 16:41:47.338878 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:47.338848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-7rwks" event={"ID":"5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6","Type":"ContainerStarted","Data":"118ea7a8abc6efea933263f6456b5d8a9d166cc2e6aaef592d90f6b1f9299068"} Jul 13 16:41:49.480399 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.480352 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerName="registry" containerID="cri-o://9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77" gracePeriod=30 Jul 13 16:41:49.746250 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.746191 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:41:49.832607 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832572 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxhs7\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832619 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832654 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832683 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832733 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832758 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.832806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832799 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.833130 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.832855 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates\") pod \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\" (UID: \"07c6f0aa-b65f-46bd-9638-14af0a076ea0\") " Jul 13 16:41:49.833423 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.833373 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:41:49.833747 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.833713 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:41:49.835322 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.835290 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:41:49.835531 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.835504 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:41:49.835618 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.835555 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7" (OuterVolumeSpecName: "kube-api-access-bxhs7") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "kube-api-access-bxhs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:41:49.835677 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.835628 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:41:49.835715 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.835680 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:41:49.841650 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.841627 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "07c6f0aa-b65f-46bd-9638-14af0a076ea0" (UID: "07c6f0aa-b65f-46bd-9638-14af0a076ea0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:41:49.934206 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934179 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934206 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934202 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/07c6f0aa-b65f-46bd-9638-14af0a076ea0-ca-trust-extracted\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934206 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934213 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-trusted-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934221 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/07c6f0aa-b65f-46bd-9638-14af0a076ea0-registry-certificates\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934231 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxhs7\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-kube-api-access-bxhs7\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934239 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07c6f0aa-b65f-46bd-9638-14af0a076ea0-bound-sa-token\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934248 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-image-registry-private-configuration\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:49.934374 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:49.934257 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/07c6f0aa-b65f-46bd-9638-14af0a076ea0-installation-pull-secrets\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:41:50.350218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.350187 2569 generic.go:356] "Generic (PLEG): container finished" podID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerID="9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77" exitCode=0 Jul 13 16:41:50.350437 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.350242 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" Jul 13 16:41:50.350437 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.350283 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" event={"ID":"07c6f0aa-b65f-46bd-9638-14af0a076ea0","Type":"ContainerDied","Data":"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77"} Jul 13 16:41:50.350437 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.350364 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8686cfbcb8-wfvbz" event={"ID":"07c6f0aa-b65f-46bd-9638-14af0a076ea0","Type":"ContainerDied","Data":"8dd56b3b0d37be79d79f01c60aa32c899dbd05bf973fe78f5828353654d60240"} Jul 13 16:41:50.350437 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.350388 2569 scope.go:117] "RemoveContainer" containerID="9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77" Jul 13 16:41:50.358867 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.358854 2569 scope.go:117] "RemoveContainer" containerID="9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77" Jul 13 16:41:50.359148 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:41:50.359128 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77\": container with ID starting with 9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77 not found: ID does not exist" containerID="9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77" Jul 13 16:41:50.359219 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.359154 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77"} err="failed to get container status \"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77\": rpc error: code = NotFound desc = could not find container \"9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77\": container with ID starting with 9cff721a90670209c3cd4568cd6d87d530baf27b92cbc8fa61a89da89e402a77 not found: ID does not exist" Jul 13 16:41:50.371096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.371067 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:41:50.375156 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:50.375136 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8686cfbcb8-wfvbz"] Jul 13 16:41:51.719253 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:51.719223 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" path="/var/lib/kubelet/pods/07c6f0aa-b65f-46bd-9638-14af0a076ea0/volumes" Jul 13 16:41:54.567130 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:54.567100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:41:54.569358 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:54.569333 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2326b3b-3b91-4d9f-957b-5a0cd8e78573-metrics-certs\") pod \"network-metrics-daemon-4r5xc\" (UID: \"a2326b3b-3b91-4d9f-957b-5a0cd8e78573\") " pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:41:54.632159 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:54.632134 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79qpb\"" Jul 13 16:41:54.640424 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:54.640403 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4r5xc" Jul 13 16:41:54.797574 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:54.797450 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4r5xc"] Jul 13 16:41:54.800251 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:41:54.800219 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2326b3b_3b91_4d9f_957b_5a0cd8e78573.slice/crio-d54e216179e090819e5ffe712d7b37e87808cb43817e0749774d7e5757a534ae WatchSource:0}: Error finding container d54e216179e090819e5ffe712d7b37e87808cb43817e0749774d7e5757a534ae: Status 404 returned error can't find the container with id d54e216179e090819e5ffe712d7b37e87808cb43817e0749774d7e5757a534ae Jul 13 16:41:55.370558 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:55.370519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r5xc" event={"ID":"a2326b3b-3b91-4d9f-957b-5a0cd8e78573","Type":"ContainerStarted","Data":"d54e216179e090819e5ffe712d7b37e87808cb43817e0749774d7e5757a534ae"} Jul 13 16:41:56.375938 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:56.375910 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r5xc" event={"ID":"a2326b3b-3b91-4d9f-957b-5a0cd8e78573","Type":"ContainerStarted","Data":"f340a0bdbf786e43f593ab1eb25b8bde1c35fa8c061c182edfc77961a5942f66"} Jul 13 16:41:57.381288 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:57.381252 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4r5xc" event={"ID":"a2326b3b-3b91-4d9f-957b-5a0cd8e78573","Type":"ContainerStarted","Data":"1019978dc8a856bfb53f2eeb98c71a6029e90d98f301e51fdff48900117029e8"} Jul 13 16:41:57.398596 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:41:57.398547 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4r5xc" podStartSLOduration=129.993907239 podStartE2EDuration="2m11.398532465s" totalImagesPullingTime="1.404625226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:57.397542197 +0000 UTC m=+132.187630131" watchObservedRunningTime="2026-07-13 16:41:57.398532465 +0000 UTC m=+132.188620398" Jul 13 16:42:01.401049 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:01.401012 2569 generic.go:356] "Generic (PLEG): container finished" podID="680fabb0-064a-4fc1-89f9-53d8c010bdd0" containerID="4a682684afbe6707185b0129b01683670657372c92c4e5a66405c0c005933f40" exitCode=0 Jul 13 16:42:01.401474 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:01.401085 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" event={"ID":"680fabb0-064a-4fc1-89f9-53d8c010bdd0","Type":"ContainerDied","Data":"4a682684afbe6707185b0129b01683670657372c92c4e5a66405c0c005933f40"} Jul 13 16:42:01.401474 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:01.401407 2569 scope.go:117] "RemoveContainer" containerID="4a682684afbe6707185b0129b01683670657372c92c4e5a66405c0c005933f40" Jul 13 16:42:02.405944 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:02.405903 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-thtld" event={"ID":"680fabb0-064a-4fc1-89f9-53d8c010bdd0","Type":"ContainerStarted","Data":"d2ea8816ca7ae168b06438927239eb66f571cd689f119930ea575e847dc3f9a4"} Jul 13 16:42:49.535908 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.535876 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:42:49.536422 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536334 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerName="registry" Jul 13 16:42:49.536422 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536354 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerName="registry" Jul 13 16:42:49.536422 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536384 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" containerName="console" Jul 13 16:42:49.536422 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536392 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" containerName="console" Jul 13 16:42:49.536624 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536467 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c6f0aa-b65f-46bd-9638-14af0a076ea0" containerName="registry" Jul 13 16:42:49.536624 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.536482 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6697f218-00b3-46af-842f-9e0faf6395e4" containerName="console" Jul 13 16:42:49.539283 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.539262 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.544623 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.544563 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 16:42:49.544847 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.544822 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 16:42:49.544925 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.544864 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8bczz\"" Jul 13 16:42:49.545325 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.545241 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 16:42:49.545325 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.545251 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 16:42:49.545699 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.545682 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 16:42:49.550889 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.550868 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 16:42:49.555567 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.555545 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:42:49.587324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587344 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587427 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587577 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587456 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587577 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587497 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zljx9\" (UniqueName: \"kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.587577 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.587550 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688201 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688401 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688331 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688401 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688374 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688401 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688570 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688513 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zljx9\" (UniqueName: \"kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688638 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688616 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.688709 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.688695 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.689105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.689085 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.689267 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.689245 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.689402 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.689378 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.689488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.689469 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.690763 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.690745 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.690877 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.690856 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.696761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.696739 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zljx9\" (UniqueName: \"kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9\") pod \"console-7485f874c-6ng2z\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.850565 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.850480 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:42:49.987638 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:49.987606 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:42:49.990475 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:42:49.990433 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312422b1_a9a0_4dae_91a5_6de991be4332.slice/crio-41766b8073aa4347124224ecb706b398f0ad69eb5def854d117ae079e5f6462d WatchSource:0}: Error finding container 41766b8073aa4347124224ecb706b398f0ad69eb5def854d117ae079e5f6462d: Status 404 returned error can't find the container with id 41766b8073aa4347124224ecb706b398f0ad69eb5def854d117ae079e5f6462d Jul 13 16:42:50.583221 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:50.583184 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7485f874c-6ng2z" event={"ID":"312422b1-a9a0-4dae-91a5-6de991be4332","Type":"ContainerStarted","Data":"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812"} Jul 13 16:42:50.583607 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:50.583238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7485f874c-6ng2z" event={"ID":"312422b1-a9a0-4dae-91a5-6de991be4332","Type":"ContainerStarted","Data":"41766b8073aa4347124224ecb706b398f0ad69eb5def854d117ae079e5f6462d"} Jul 13 16:42:50.604763 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:50.604717 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7485f874c-6ng2z" podStartSLOduration=1.604702197 podStartE2EDuration="1.604702197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:42:50.602577161 +0000 UTC m=+185.392665094" watchObservedRunningTime="2026-07-13 16:42:50.604702197 +0000 UTC m=+185.394790130" Jul 13 16:42:56.762148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.762110 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:42:56.789473 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.789443 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:42:56.791759 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.791738 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.805067 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.805021 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:42:56.838682 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838657 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.838831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838695 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.838831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838716 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.838831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838773 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.838831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838800 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.838831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.839004 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.838874 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxxxv\" (UniqueName: \"kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939627 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939595 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxxxv\" (UniqueName: \"kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939634 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939657 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939677 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939720 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939755 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.939987 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.939894 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.940553 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.940532 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.940673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.940556 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.940673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.940613 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.940673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.940651 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.942162 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.942144 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.942412 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.942390 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:56.948443 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:56.948422 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxxxv\" (UniqueName: \"kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv\") pod \"console-56f64ccdf7-jx2hn\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:57.100805 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:57.100718 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:42:57.228922 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:57.228896 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:42:57.230974 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:42:57.230944 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26458d20_2395_4cab_975f_8d531d83374a.slice/crio-be73955777cd4d7d7bec0e4e616bdf0b2e312f27c6425e8d0d9bcd8bb6187f2d WatchSource:0}: Error finding container be73955777cd4d7d7bec0e4e616bdf0b2e312f27c6425e8d0d9bcd8bb6187f2d: Status 404 returned error can't find the container with id be73955777cd4d7d7bec0e4e616bdf0b2e312f27c6425e8d0d9bcd8bb6187f2d Jul 13 16:42:57.609521 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:57.609489 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f64ccdf7-jx2hn" event={"ID":"26458d20-2395-4cab-975f-8d531d83374a","Type":"ContainerStarted","Data":"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44"} Jul 13 16:42:57.609682 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:57.609535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f64ccdf7-jx2hn" event={"ID":"26458d20-2395-4cab-975f-8d531d83374a","Type":"ContainerStarted","Data":"be73955777cd4d7d7bec0e4e616bdf0b2e312f27c6425e8d0d9bcd8bb6187f2d"} Jul 13 16:42:57.628177 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:57.628126 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f64ccdf7-jx2hn" podStartSLOduration=1.628112029 podStartE2EDuration="1.628112029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:42:57.627016251 +0000 UTC m=+192.417104184" watchObservedRunningTime="2026-07-13 16:42:57.628112029 +0000 UTC m=+192.418199963" Jul 13 16:42:59.850920 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:42:59.850891 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:43:07.101869 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:07.101838 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:43:07.102353 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:07.101898 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:43:07.106497 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:07.106474 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:43:07.651673 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:07.651642 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:43:21.782341 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:21.782281 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7485f874c-6ng2z" podUID="312422b1-a9a0-4dae-91a5-6de991be4332" containerName="console" containerID="cri-o://7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812" gracePeriod=15 Jul 13 16:43:22.016905 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.016879 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:43:22.131771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131692 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.131771 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131744 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zljx9\" (UniqueName: \"kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131858 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131893 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131915 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131945 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.132169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.131964 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config\") pod \"312422b1-a9a0-4dae-91a5-6de991be4332\" (UID: \"312422b1-a9a0-4dae-91a5-6de991be4332\") " Jul 13 16:43:22.132279 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.132250 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:43:22.132353 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.132306 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config" (OuterVolumeSpecName: "console-config") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:43:22.132353 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.132326 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:43:22.132460 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.132377 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca" (OuterVolumeSpecName: "service-ca") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:43:22.133980 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.133956 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9" (OuterVolumeSpecName: "kube-api-access-zljx9") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "kube-api-access-zljx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:43:22.134096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.134081 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:43:22.134335 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.134307 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "312422b1-a9a0-4dae-91a5-6de991be4332" (UID: "312422b1-a9a0-4dae-91a5-6de991be4332"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:43:22.233490 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233465 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-oauth-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233490 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233489 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zljx9\" (UniqueName: \"kubernetes.io/projected/312422b1-a9a0-4dae-91a5-6de991be4332-kube-api-access-zljx9\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233499 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-trusted-ca-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233507 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-console-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233515 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/312422b1-a9a0-4dae-91a5-6de991be4332-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233523 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.233654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.233531 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/312422b1-a9a0-4dae-91a5-6de991be4332-console-oauth-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:43:22.706267 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.706228 2569 generic.go:356] "Generic (PLEG): container finished" podID="312422b1-a9a0-4dae-91a5-6de991be4332" containerID="7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812" exitCode=0 Jul 13 16:43:22.706435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.706276 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7485f874c-6ng2z" Jul 13 16:43:22.706435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.706298 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7485f874c-6ng2z" event={"ID":"312422b1-a9a0-4dae-91a5-6de991be4332","Type":"ContainerDied","Data":"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812"} Jul 13 16:43:22.706435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.706345 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7485f874c-6ng2z" event={"ID":"312422b1-a9a0-4dae-91a5-6de991be4332","Type":"ContainerDied","Data":"41766b8073aa4347124224ecb706b398f0ad69eb5def854d117ae079e5f6462d"} Jul 13 16:43:22.706435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.706371 2569 scope.go:117] "RemoveContainer" containerID="7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812" Jul 13 16:43:22.714702 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.714677 2569 scope.go:117] "RemoveContainer" containerID="7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812" Jul 13 16:43:22.714935 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:43:22.714915 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812\": container with ID starting with 7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812 not found: ID does not exist" containerID="7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812" Jul 13 16:43:22.714988 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.714945 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812"} err="failed to get container status \"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812\": rpc error: code = NotFound desc = could not find container \"7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812\": container with ID starting with 7fe9ad736590f00cc679f6498255bc580738c4ba0ece726c870dd195c61d3812 not found: ID does not exist" Jul 13 16:43:22.727642 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.727619 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:43:22.731744 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:22.731722 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7485f874c-6ng2z"] Jul 13 16:43:23.718372 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:43:23.718343 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="312422b1-a9a0-4dae-91a5-6de991be4332" path="/var/lib/kubelet/pods/312422b1-a9a0-4dae-91a5-6de991be4332/volumes" Jul 13 16:44:14.392160 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.392072 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc"] Jul 13 16:44:14.392630 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.392476 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312422b1-a9a0-4dae-91a5-6de991be4332" containerName="console" Jul 13 16:44:14.392630 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.392494 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="312422b1-a9a0-4dae-91a5-6de991be4332" containerName="console" Jul 13 16:44:14.392630 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.392585 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="312422b1-a9a0-4dae-91a5-6de991be4332" containerName="console" Jul 13 16:44:14.394666 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.394645 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.397441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.397420 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 16:44:14.397535 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.397438 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 16:44:14.398543 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.398523 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bn4wf\"" Jul 13 16:44:14.406498 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.406474 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc"] Jul 13 16:44:14.423631 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.423606 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.423744 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.423645 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdttm\" (UniqueName: \"kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.423744 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.423666 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.524587 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.524561 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.524713 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.524600 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdttm\" (UniqueName: \"kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.524713 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.524622 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.524940 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.524921 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.525010 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.524993 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.533027 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.533003 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdttm\" (UniqueName: \"kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.703459 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.703429 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:14.837057 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.836979 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc"] Jul 13 16:44:14.839369 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:44:14.839341 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15ec924_4582_4a41_96da_27e2eb8e235d.slice/crio-3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f WatchSource:0}: Error finding container 3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f: Status 404 returned error can't find the container with id 3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f Jul 13 16:44:14.890127 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:14.890090 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerStarted","Data":"3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f"} Jul 13 16:44:22.923918 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:22.923880 2569 generic.go:356] "Generic (PLEG): container finished" podID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerID="6397c651a9b8983910ce78982514fa3475555460b2b2ce1fe9095080d6e7f4d1" exitCode=0 Jul 13 16:44:22.924352 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:22.923975 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerDied","Data":"6397c651a9b8983910ce78982514fa3475555460b2b2ce1fe9095080d6e7f4d1"} Jul 13 16:44:24.934304 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:24.934276 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerStarted","Data":"3dafd1b887249c155f5e2a7b32663a68ed7e6d739beb2d3b5bd5aa43d9598908"} Jul 13 16:44:25.939435 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:25.939401 2569 generic.go:356] "Generic (PLEG): container finished" podID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerID="3dafd1b887249c155f5e2a7b32663a68ed7e6d739beb2d3b5bd5aa43d9598908" exitCode=0 Jul 13 16:44:25.939832 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:25.939491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerDied","Data":"3dafd1b887249c155f5e2a7b32663a68ed7e6d739beb2d3b5bd5aa43d9598908"} Jul 13 16:44:33.977902 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:33.977868 2569 generic.go:356] "Generic (PLEG): container finished" podID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerID="9278a28702a6e721ad47ac1b35356eefbf5e34bb803d643f83f9b405bf8236a8" exitCode=0 Jul 13 16:44:33.978315 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:33.977950 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerDied","Data":"9278a28702a6e721ad47ac1b35356eefbf5e34bb803d643f83f9b405bf8236a8"} Jul 13 16:44:35.145997 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.145971 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:35.200819 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.200787 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util\") pod \"e15ec924-4582-4a41-96da-27e2eb8e235d\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " Jul 13 16:44:35.200955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.200880 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle\") pod \"e15ec924-4582-4a41-96da-27e2eb8e235d\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " Jul 13 16:44:35.200955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.200905 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdttm\" (UniqueName: \"kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm\") pod \"e15ec924-4582-4a41-96da-27e2eb8e235d\" (UID: \"e15ec924-4582-4a41-96da-27e2eb8e235d\") " Jul 13 16:44:35.201508 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.201481 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle" (OuterVolumeSpecName: "bundle") pod "e15ec924-4582-4a41-96da-27e2eb8e235d" (UID: "e15ec924-4582-4a41-96da-27e2eb8e235d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:44:35.203066 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.203024 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm" (OuterVolumeSpecName: "kube-api-access-qdttm") pod "e15ec924-4582-4a41-96da-27e2eb8e235d" (UID: "e15ec924-4582-4a41-96da-27e2eb8e235d"). InnerVolumeSpecName "kube-api-access-qdttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:44:35.204901 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.204883 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util" (OuterVolumeSpecName: "util") pod "e15ec924-4582-4a41-96da-27e2eb8e235d" (UID: "e15ec924-4582-4a41-96da-27e2eb8e235d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:44:35.301812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.301745 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:35.301812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.301769 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdttm\" (UniqueName: \"kubernetes.io/projected/e15ec924-4582-4a41-96da-27e2eb8e235d-kube-api-access-qdttm\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:35.301812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.301779 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e15ec924-4582-4a41-96da-27e2eb8e235d-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:35.987149 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.987074 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" Jul 13 16:44:35.987300 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.987075 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w7vhc" event={"ID":"e15ec924-4582-4a41-96da-27e2eb8e235d","Type":"ContainerDied","Data":"3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f"} Jul 13 16:44:35.987300 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:35.987192 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3672a5832bef9e7521287e0bcececcb67d3e6101f263fbc927395108009c122f" Jul 13 16:44:42.423812 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.423775 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2"] Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424065 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="pull" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424080 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="pull" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424106 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="util" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424112 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="util" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424128 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="extract" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424134 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="extract" Jul 13 16:44:42.424222 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.424182 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15ec924-4582-4a41-96da-27e2eb8e235d" containerName="extract" Jul 13 16:44:42.425933 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.425917 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.428619 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.428590 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:44:42.428746 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.428694 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 16:44:42.429002 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.428983 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l5xzz\"" Jul 13 16:44:42.441444 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.441420 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2"] Jul 13 16:44:42.558641 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.558602 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2717d8b1-21c5-4789-aea0-8e0d2e291770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.558641 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.558642 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49z59\" (UniqueName: \"kubernetes.io/projected/2717d8b1-21c5-4789-aea0-8e0d2e291770-kube-api-access-49z59\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.658972 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.658938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2717d8b1-21c5-4789-aea0-8e0d2e291770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.659135 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.658982 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49z59\" (UniqueName: \"kubernetes.io/projected/2717d8b1-21c5-4789-aea0-8e0d2e291770-kube-api-access-49z59\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.659329 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.659309 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2717d8b1-21c5-4789-aea0-8e0d2e291770-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.671434 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.671402 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49z59\" (UniqueName: \"kubernetes.io/projected/2717d8b1-21c5-4789-aea0-8e0d2e291770-kube-api-access-49z59\") pod \"cert-manager-operator-controller-manager-55c444bf86-rzvj2\" (UID: \"2717d8b1-21c5-4789-aea0-8e0d2e291770\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.735367 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.735297 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" Jul 13 16:44:42.868196 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:42.868170 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2"] Jul 13 16:44:42.871055 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:44:42.871013 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2717d8b1_21c5_4789_aea0_8e0d2e291770.slice/crio-a80c3a71d2f61ace5ebc8caa40111a1e2adaee9c25f0a74e603ef6af7cc034d9 WatchSource:0}: Error finding container a80c3a71d2f61ace5ebc8caa40111a1e2adaee9c25f0a74e603ef6af7cc034d9: Status 404 returned error can't find the container with id a80c3a71d2f61ace5ebc8caa40111a1e2adaee9c25f0a74e603ef6af7cc034d9 Jul 13 16:44:43.014776 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:43.014686 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" event={"ID":"2717d8b1-21c5-4789-aea0-8e0d2e291770","Type":"ContainerStarted","Data":"a80c3a71d2f61ace5ebc8caa40111a1e2adaee9c25f0a74e603ef6af7cc034d9"} Jul 13 16:44:45.645654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:45.645626 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:44:45.649701 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:45.649677 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:44:45.654371 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:45.654347 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:44:45.657436 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:45.657420 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 16:44:45.659104 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:45.659086 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:44:46.030517 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.030482 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" event={"ID":"2717d8b1-21c5-4789-aea0-8e0d2e291770","Type":"ContainerStarted","Data":"4fcf3df20a08ed6fc620709c102393b5d220cec8b45acd2b1d89d474694639ab"} Jul 13 16:44:46.054361 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.054307 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rzvj2" podStartSLOduration=1.389352278 podStartE2EDuration="4.054293156s" totalImagesPullingTime="2.664940878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:44:46.052779089 +0000 UTC m=+300.842867023" watchObservedRunningTime="2026-07-13 16:44:46.054293156 +0000 UTC m=+300.844381086" Jul 13 16:44:46.809097 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.809060 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m"] Jul 13 16:44:46.811477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.811459 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.816858 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.816835 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 16:44:46.817103 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.817085 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bn4wf\"" Jul 13 16:44:46.817189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.817170 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 16:44:46.837802 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.837774 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m"] Jul 13 16:44:46.890554 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.890526 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.890695 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.890565 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwr2d\" (UniqueName: \"kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.890695 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.890653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.991846 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.991810 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.992013 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.991852 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwr2d\" (UniqueName: \"kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.992013 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.991909 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.992308 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.992285 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:46.992359 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:46.992295 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:47.009844 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:47.009814 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwr2d\" (UniqueName: \"kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:47.119995 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:47.119924 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:47.246380 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:47.246248 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m"] Jul 13 16:44:47.249122 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:44:47.249091 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e24067_d108_4e0d_aaed_6a9cf165448b.slice/crio-fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0 WatchSource:0}: Error finding container fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0: Status 404 returned error can't find the container with id fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0 Jul 13 16:44:48.039507 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:48.039475 2569 generic.go:356] "Generic (PLEG): container finished" podID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerID="12efd481c4a9f096062e2268baca76594293511c216130fb5185e4be080ca9c9" exitCode=0 Jul 13 16:44:48.039831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:48.039563 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" event={"ID":"89e24067-d108-4e0d-aaed-6a9cf165448b","Type":"ContainerDied","Data":"12efd481c4a9f096062e2268baca76594293511c216130fb5185e4be080ca9c9"} Jul 13 16:44:48.039831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:48.039607 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" event={"ID":"89e24067-d108-4e0d-aaed-6a9cf165448b","Type":"ContainerStarted","Data":"fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0"} Jul 13 16:44:48.040482 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:48.040466 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 16:44:50.800263 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.800229 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b98cb"] Jul 13 16:44:50.802448 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.802432 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:50.805296 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.805273 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z5pqs\"" Jul 13 16:44:50.805423 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.805273 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 16:44:50.806265 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.806248 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 16:44:50.811668 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.811647 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b98cb"] Jul 13 16:44:50.925368 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.925325 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtj6s\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-kube-api-access-vtj6s\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:50.925527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:50.925425 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.026858 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.026823 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtj6s\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-kube-api-access-vtj6s\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.027030 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.026881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.034981 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.034950 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.035122 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.035016 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtj6s\" (UniqueName: \"kubernetes.io/projected/a7948da1-468a-40bb-9c71-89833dab7747-kube-api-access-vtj6s\") pod \"cert-manager-cainjector-559cdd8c87-b98cb\" (UID: \"a7948da1-468a-40bb-9c71-89833dab7747\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.055416 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.055346 2569 generic.go:356] "Generic (PLEG): container finished" podID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerID="76ac87de7f5ae0ee92a2de0ab49a8ab7227224beeec19184d24a5a4e7eea51f8" exitCode=0 Jul 13 16:44:51.055539 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.055428 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" event={"ID":"89e24067-d108-4e0d-aaed-6a9cf165448b","Type":"ContainerDied","Data":"76ac87de7f5ae0ee92a2de0ab49a8ab7227224beeec19184d24a5a4e7eea51f8"} Jul 13 16:44:51.128105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.128081 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" Jul 13 16:44:51.257446 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:51.257422 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b98cb"] Jul 13 16:44:51.259892 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:44:51.259867 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7948da1_468a_40bb_9c71_89833dab7747.slice/crio-33aa9a54857b10afed2a38b34a47aa09a961859bf6aedcbed1422f672ae8fc06 WatchSource:0}: Error finding container 33aa9a54857b10afed2a38b34a47aa09a961859bf6aedcbed1422f672ae8fc06: Status 404 returned error can't find the container with id 33aa9a54857b10afed2a38b34a47aa09a961859bf6aedcbed1422f672ae8fc06 Jul 13 16:44:52.063309 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:52.063276 2569 generic.go:356] "Generic (PLEG): container finished" podID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerID="56d4fd3011b244f98e14925389b1da9be764fe1bbec98ddf67b480291e08bafb" exitCode=0 Jul 13 16:44:52.063769 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:52.063365 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" event={"ID":"89e24067-d108-4e0d-aaed-6a9cf165448b","Type":"ContainerDied","Data":"56d4fd3011b244f98e14925389b1da9be764fe1bbec98ddf67b480291e08bafb"} Jul 13 16:44:52.064799 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:52.064770 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" event={"ID":"a7948da1-468a-40bb-9c71-89833dab7747","Type":"ContainerStarted","Data":"33aa9a54857b10afed2a38b34a47aa09a961859bf6aedcbed1422f672ae8fc06"} Jul 13 16:44:53.712292 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.712266 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:53.851389 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.851356 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle\") pod \"89e24067-d108-4e0d-aaed-6a9cf165448b\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " Jul 13 16:44:53.851594 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.851414 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwr2d\" (UniqueName: \"kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d\") pod \"89e24067-d108-4e0d-aaed-6a9cf165448b\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " Jul 13 16:44:53.851594 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.851433 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util\") pod \"89e24067-d108-4e0d-aaed-6a9cf165448b\" (UID: \"89e24067-d108-4e0d-aaed-6a9cf165448b\") " Jul 13 16:44:53.851881 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.851839 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle" (OuterVolumeSpecName: "bundle") pod "89e24067-d108-4e0d-aaed-6a9cf165448b" (UID: "89e24067-d108-4e0d-aaed-6a9cf165448b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:44:53.854001 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.853971 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d" (OuterVolumeSpecName: "kube-api-access-wwr2d") pod "89e24067-d108-4e0d-aaed-6a9cf165448b" (UID: "89e24067-d108-4e0d-aaed-6a9cf165448b"). InnerVolumeSpecName "kube-api-access-wwr2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:44:53.856610 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.856588 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util" (OuterVolumeSpecName: "util") pod "89e24067-d108-4e0d-aaed-6a9cf165448b" (UID: "89e24067-d108-4e0d-aaed-6a9cf165448b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:44:53.953070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.952925 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:53.953070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.952973 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwr2d\" (UniqueName: \"kubernetes.io/projected/89e24067-d108-4e0d-aaed-6a9cf165448b-kube-api-access-wwr2d\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:53.953070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:53.952989 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89e24067-d108-4e0d-aaed-6a9cf165448b-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:44:54.076012 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:54.075979 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" Jul 13 16:44:54.076213 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:54.076000 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fngc5m" event={"ID":"89e24067-d108-4e0d-aaed-6a9cf165448b","Type":"ContainerDied","Data":"fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0"} Jul 13 16:44:54.076213 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:54.076079 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbef3eb0aed69b81677947cc2acd343df8dc334b32562607edfc488223e2a5d0" Jul 13 16:44:54.077759 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:54.077713 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" event={"ID":"a7948da1-468a-40bb-9c71-89833dab7747","Type":"ContainerStarted","Data":"55808a8531609044cdbac0b30b50b654e105e43592f6fe49433efe6ab60faf1b"} Jul 13 16:44:54.093718 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:44:54.093678 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b98cb" podStartSLOduration=1.595977548 podStartE2EDuration="4.093662339s" totalImagesPullingTime="2.497684791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:44:54.093506967 +0000 UTC m=+308.883594914" watchObservedRunningTime="2026-07-13 16:44:54.093662339 +0000 UTC m=+308.883750273" Jul 13 16:45:01.389106 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389067 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj"] Jul 13 16:45:01.389571 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389537 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="util" Jul 13 16:45:01.389571 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389556 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="util" Jul 13 16:45:01.389683 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389583 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="extract" Jul 13 16:45:01.389683 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389602 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="extract" Jul 13 16:45:01.389683 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389621 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="pull" Jul 13 16:45:01.389683 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389629 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="pull" Jul 13 16:45:01.389858 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.389702 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e24067-d108-4e0d-aaed-6a9cf165448b" containerName="extract" Jul 13 16:45:01.395392 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.395367 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.398229 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.398204 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 16:45:01.398322 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.398291 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 16:45:01.399232 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.399216 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7q5k2\"" Jul 13 16:45:01.406630 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.406609 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj"] Jul 13 16:45:01.513504 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.513476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7wm\" (UniqueName: \"kubernetes.io/projected/02dd630a-f3f0-438c-a623-9ee603c74520-kube-api-access-lz7wm\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.513642 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.513539 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02dd630a-f3f0-438c-a623-9ee603c74520-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.614549 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.614515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02dd630a-f3f0-438c-a623-9ee603c74520-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.614700 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.614579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7wm\" (UniqueName: \"kubernetes.io/projected/02dd630a-f3f0-438c-a623-9ee603c74520-kube-api-access-lz7wm\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.614978 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.614948 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02dd630a-f3f0-438c-a623-9ee603c74520-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.623627 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.623593 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7wm\" (UniqueName: \"kubernetes.io/projected/02dd630a-f3f0-438c-a623-9ee603c74520-kube-api-access-lz7wm\") pod \"openshift-lws-operator-fd8ccff4c-bltsj\" (UID: \"02dd630a-f3f0-438c-a623-9ee603c74520\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.704518 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.704483 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" Jul 13 16:45:01.839689 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:01.839666 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj"] Jul 13 16:45:01.842231 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:45:01.842198 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02dd630a_f3f0_438c_a623_9ee603c74520.slice/crio-65fc2c79cd07e77e46d057df4bfa23b93ed7a51ddcefd4978c31ac5517aa4d27 WatchSource:0}: Error finding container 65fc2c79cd07e77e46d057df4bfa23b93ed7a51ddcefd4978c31ac5517aa4d27: Status 404 returned error can't find the container with id 65fc2c79cd07e77e46d057df4bfa23b93ed7a51ddcefd4978c31ac5517aa4d27 Jul 13 16:45:02.113857 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:02.113772 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" event={"ID":"02dd630a-f3f0-438c-a623-9ee603c74520","Type":"ContainerStarted","Data":"65fc2c79cd07e77e46d057df4bfa23b93ed7a51ddcefd4978c31ac5517aa4d27"} Jul 13 16:45:05.130568 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:05.130526 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" event={"ID":"02dd630a-f3f0-438c-a623-9ee603c74520","Type":"ContainerStarted","Data":"32742d800bd77da7e41bc98725d51c07bb46cb609a99ff083ef94bb50ddbdd09"} Jul 13 16:45:05.147945 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:05.147892 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bltsj" podStartSLOduration=1.075289508 podStartE2EDuration="4.147877058s" totalImagesPullingTime="3.07258755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:05.14702409 +0000 UTC m=+319.937112025" watchObservedRunningTime="2026-07-13 16:45:05.147877058 +0000 UTC m=+319.937964992" Jul 13 16:45:07.594651 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.594619 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq"] Jul 13 16:45:07.613913 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.613882 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq"] Jul 13 16:45:07.614093 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.614010 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.617142 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.617115 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 16:45:07.618075 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.618056 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 16:45:07.618173 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.618139 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bn4wf\"" Jul 13 16:45:07.663254 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.663223 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.663427 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.663270 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.663427 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.663292 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbpch\" (UniqueName: \"kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.763941 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.763904 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.764146 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.763966 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.764146 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.763993 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbpch\" (UniqueName: \"kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.764344 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.764318 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.764448 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.764346 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.771803 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.771783 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbpch\" (UniqueName: \"kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:07.924739 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:07.924713 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:08.099307 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:08.099236 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq"] Jul 13 16:45:08.103305 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:45:08.103251 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4439a67c_3edd_48fe_8b07_fa1f1fd6e70e.slice/crio-31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb WatchSource:0}: Error finding container 31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb: Status 404 returned error can't find the container with id 31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb Jul 13 16:45:08.146212 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:08.146169 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" event={"ID":"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e","Type":"ContainerStarted","Data":"31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb"} Jul 13 16:45:09.153104 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:09.153064 2569 generic.go:356] "Generic (PLEG): container finished" podID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerID="c3ff21648263a83ea4676074c922c2ff17705c00a9d59faf5fb74d2bf6866f01" exitCode=0 Jul 13 16:45:09.153488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:09.153152 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" event={"ID":"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e","Type":"ContainerDied","Data":"c3ff21648263a83ea4676074c922c2ff17705c00a9d59faf5fb74d2bf6866f01"} Jul 13 16:45:11.163922 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:11.163888 2569 generic.go:356] "Generic (PLEG): container finished" podID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerID="80eb32f3af0574919c9f67d1d52b06ac47085f663db3c2599717e18d24c7ff25" exitCode=0 Jul 13 16:45:11.164326 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:11.163968 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" event={"ID":"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e","Type":"ContainerDied","Data":"80eb32f3af0574919c9f67d1d52b06ac47085f663db3c2599717e18d24c7ff25"} Jul 13 16:45:12.170403 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:12.170371 2569 generic.go:356] "Generic (PLEG): container finished" podID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerID="a578eb0ee7b57de580fb0dfd6aefbdfec7a9bfa7e915486de6f0a9577c3692a6" exitCode=0 Jul 13 16:45:12.170802 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:12.170453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" event={"ID":"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e","Type":"ContainerDied","Data":"a578eb0ee7b57de580fb0dfd6aefbdfec7a9bfa7e915486de6f0a9577c3692a6"} Jul 13 16:45:13.299362 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.299340 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:13.318125 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.318094 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle\") pod \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " Jul 13 16:45:13.318268 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.318140 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbpch\" (UniqueName: \"kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch\") pod \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " Jul 13 16:45:13.318268 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.318194 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util\") pod \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\" (UID: \"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e\") " Jul 13 16:45:13.318976 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.318945 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle" (OuterVolumeSpecName: "bundle") pod "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" (UID: "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:45:13.320684 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.320655 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch" (OuterVolumeSpecName: "kube-api-access-gbpch") pod "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" (UID: "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e"). InnerVolumeSpecName "kube-api-access-gbpch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:45:13.325107 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.325066 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util" (OuterVolumeSpecName: "util") pod "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" (UID: "4439a67c-3edd-48fe-8b07-fa1f1fd6e70e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:45:13.418769 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.418735 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:45:13.418769 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.418762 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbpch\" (UniqueName: \"kubernetes.io/projected/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-kube-api-access-gbpch\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:45:13.418769 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:13.418771 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4439a67c-3edd-48fe-8b07-fa1f1fd6e70e-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:45:14.181351 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:14.181323 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" Jul 13 16:45:14.181351 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:14.181329 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftnrnq" event={"ID":"4439a67c-3edd-48fe-8b07-fa1f1fd6e70e","Type":"ContainerDied","Data":"31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb"} Jul 13 16:45:14.181549 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:14.181372 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31dd426b21479eed62c9285373f5ac5a746f54d9c47edb095c3116213293aadb" Jul 13 16:45:16.431388 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431351 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm"] Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431637 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="util" Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431648 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="util" Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431660 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="extract" Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431668 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="extract" Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431692 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="pull" Jul 13 16:45:16.431748 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431701 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="pull" Jul 13 16:45:16.431927 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.431763 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4439a67c-3edd-48fe-8b07-fa1f1fd6e70e" containerName="extract" Jul 13 16:45:16.433794 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.433778 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.437469 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.437447 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 16:45:16.437574 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.437469 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 16:45:16.437574 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.437469 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 16:45:16.437574 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.437447 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8lrhf\"" Jul 13 16:45:16.440849 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.440826 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61ac3113-0bf3-43fc-bcb0-bc617d8210df-manager-config\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.440939 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.440877 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.441000 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.440969 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.441084 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.441064 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhdlh\" (UniqueName: \"kubernetes.io/projected/61ac3113-0bf3-43fc-bcb0-bc617d8210df-kube-api-access-mhdlh\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.448323 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.448300 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm"] Jul 13 16:45:16.542151 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.542119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.542333 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.542181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.542333 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.542246 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhdlh\" (UniqueName: \"kubernetes.io/projected/61ac3113-0bf3-43fc-bcb0-bc617d8210df-kube-api-access-mhdlh\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.542333 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.542285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61ac3113-0bf3-43fc-bcb0-bc617d8210df-manager-config\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.542871 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.542846 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/61ac3113-0bf3-43fc-bcb0-bc617d8210df-manager-config\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.544610 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.544576 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.544610 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.544588 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61ac3113-0bf3-43fc-bcb0-bc617d8210df-cert\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.561654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.561631 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhdlh\" (UniqueName: \"kubernetes.io/projected/61ac3113-0bf3-43fc-bcb0-bc617d8210df-kube-api-access-mhdlh\") pod \"lws-controller-manager-78cc96c4cb-nwcsm\" (UID: \"61ac3113-0bf3-43fc-bcb0-bc617d8210df\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.743421 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.743344 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:16.880566 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:16.880544 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm"] Jul 13 16:45:16.882649 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:45:16.882620 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ac3113_0bf3_43fc_bcb0_bc617d8210df.slice/crio-d6413c0e8640d5ee9e0135dc3d8ee178dd26021e39930569c1fa99661591c2cf WatchSource:0}: Error finding container d6413c0e8640d5ee9e0135dc3d8ee178dd26021e39930569c1fa99661591c2cf: Status 404 returned error can't find the container with id d6413c0e8640d5ee9e0135dc3d8ee178dd26021e39930569c1fa99661591c2cf Jul 13 16:45:17.195481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:17.195439 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" event={"ID":"61ac3113-0bf3-43fc-bcb0-bc617d8210df","Type":"ContainerStarted","Data":"d6413c0e8640d5ee9e0135dc3d8ee178dd26021e39930569c1fa99661591c2cf"} Jul 13 16:45:24.920160 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.920127 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds"] Jul 13 16:45:24.923248 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.923226 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:24.930721 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.930705 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 16:45:24.930920 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.930903 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 16:45:24.931021 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.930968 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 16:45:24.931021 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.930968 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 16:45:24.931137 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.931079 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h2scv\"" Jul 13 16:45:24.959942 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:24.959913 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds"] Jul 13 16:45:25.013404 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.013373 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.013552 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.013418 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.013552 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.013469 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6dlj\" (UniqueName: \"kubernetes.io/projected/df850945-13c1-43e6-af5b-953b50c9f720-kube-api-access-s6dlj\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.013552 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.013522 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.013661 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.013552 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114146 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114109 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114287 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114164 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114287 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114196 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114287 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114218 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6dlj\" (UniqueName: \"kubernetes.io/projected/df850945-13c1-43e6-af5b-953b50c9f720-kube-api-access-s6dlj\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114287 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.114653 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.114632 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.116463 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.116431 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df850945-13c1-43e6-af5b-953b50c9f720-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.116770 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.116748 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.116770 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.116763 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df850945-13c1-43e6-af5b-953b50c9f720-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.128303 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.128281 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6dlj\" (UniqueName: \"kubernetes.io/projected/df850945-13c1-43e6-af5b-953b50c9f720-kube-api-access-s6dlj\") pod \"opendatahub-operator-controller-manager-5857456dfc-xk7ds\" (UID: \"df850945-13c1-43e6-af5b-953b50c9f720\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.233586 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.233501 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:25.368536 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:25.368511 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds"] Jul 13 16:45:25.371023 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:45:25.370998 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf850945_13c1_43e6_af5b_953b50c9f720.slice/crio-6018f1c23882239ecc1b4108a6d1247558be9f8b7c80ed6e5228ffa3c8c0e4c1 WatchSource:0}: Error finding container 6018f1c23882239ecc1b4108a6d1247558be9f8b7c80ed6e5228ffa3c8c0e4c1: Status 404 returned error can't find the container with id 6018f1c23882239ecc1b4108a6d1247558be9f8b7c80ed6e5228ffa3c8c0e4c1 Jul 13 16:45:26.239429 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:26.239385 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" event={"ID":"df850945-13c1-43e6-af5b-953b50c9f720","Type":"ContainerStarted","Data":"6018f1c23882239ecc1b4108a6d1247558be9f8b7c80ed6e5228ffa3c8c0e4c1"} Jul 13 16:45:29.255556 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:29.255519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" event={"ID":"df850945-13c1-43e6-af5b-953b50c9f720","Type":"ContainerStarted","Data":"04b0e3137b9c960500da42b397bed0ce650b630edc199ecba394a91d9247d00b"} Jul 13 16:45:30.261917 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:30.261884 2569 generic.go:356] "Generic (PLEG): container finished" podID="df850945-13c1-43e6-af5b-953b50c9f720" containerID="04b0e3137b9c960500da42b397bed0ce650b630edc199ecba394a91d9247d00b" exitCode=0 Jul 13 16:45:30.262332 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:30.261972 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" event={"ID":"df850945-13c1-43e6-af5b-953b50c9f720","Type":"ContainerDied","Data":"04b0e3137b9c960500da42b397bed0ce650b630edc199ecba394a91d9247d00b"} Jul 13 16:45:31.268482 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:31.268443 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" event={"ID":"df850945-13c1-43e6-af5b-953b50c9f720","Type":"ContainerStarted","Data":"686b36f33ef0a4ef5246b8965b0c670d5c5e5bf7622ba0e40273ada519e0cc42"} Jul 13 16:45:31.268952 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:31.268642 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:31.292311 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:31.292259 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" podStartSLOduration=2.684212153 podStartE2EDuration="7.292243166s" totalImagesPullingTime="3.590067132s" totalInitContainerRuntime="1.017963881s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:31.289215689 +0000 UTC m=+346.079303623" watchObservedRunningTime="2026-07-13 16:45:31.292243166 +0000 UTC m=+346.082331099" Jul 13 16:45:32.274189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:32.274100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" event={"ID":"61ac3113-0bf3-43fc-bcb0-bc617d8210df","Type":"ContainerStarted","Data":"480536a9600a69977b31b33c78810d3421f3cc9809516f05b1af5f84fa61a58a"} Jul 13 16:45:32.274573 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:32.274392 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:32.293336 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:32.293288 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" podStartSLOduration=1.157287203 podStartE2EDuration="16.293268488s" totalImagesPullingTime="15.135981285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:32.292366062 +0000 UTC m=+347.082453996" watchObservedRunningTime="2026-07-13 16:45:32.293268488 +0000 UTC m=+347.083356422" Jul 13 16:45:42.276831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:42.276750 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-xk7ds" Jul 13 16:45:43.280655 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:43.280616 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-nwcsm" Jul 13 16:45:52.864787 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.864746 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86"] Jul 13 16:45:52.869817 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.869798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.872632 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.872610 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 16:45:52.872632 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.872627 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 16:45:52.872823 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.872742 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 16:45:52.872823 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.872767 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6745p\"" Jul 13 16:45:52.872933 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.872873 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 16:45:52.883098 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.883071 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86"] Jul 13 16:45:52.950527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950486 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950527 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950738 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950738 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950625 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950738 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950738 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950713 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdj5s\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-kube-api-access-bdj5s\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:52.950738 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:52.950734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded7ff20-866f-43ec-9815-3a3712ffea91-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.051918 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.051878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.051924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.051952 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdj5s\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-kube-api-access-bdj5s\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.051998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded7ff20-866f-43ec-9815-3a3712ffea91-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.052082 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052123 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.052118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052379 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.052154 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.052789 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.052757 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.054493 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.054457 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ded7ff20-866f-43ec-9815-3a3712ffea91-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.054590 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.054501 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.055403 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.055375 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.055519 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.055439 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.074598 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.074571 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.074874 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.074854 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdj5s\" (UniqueName: \"kubernetes.io/projected/ded7ff20-866f-43ec-9815-3a3712ffea91-kube-api-access-bdj5s\") pod \"istiod-openshift-gateway-94bb8fbfd-cgp86\" (UID: \"ded7ff20-866f-43ec-9815-3a3712ffea91\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.178885 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.178857 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:53.315951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.315918 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86"] Jul 13 16:45:53.318076 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:45:53.318032 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded7ff20_866f_43ec_9815_3a3712ffea91.slice/crio-dce1eee894bc1ec59aa3be858102e8408774caf739a183403cfa705d799f5e77 WatchSource:0}: Error finding container dce1eee894bc1ec59aa3be858102e8408774caf739a183403cfa705d799f5e77: Status 404 returned error can't find the container with id dce1eee894bc1ec59aa3be858102e8408774caf739a183403cfa705d799f5e77 Jul 13 16:45:53.374289 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:53.374257 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" event={"ID":"ded7ff20-866f-43ec-9815-3a3712ffea91","Type":"ContainerStarted","Data":"dce1eee894bc1ec59aa3be858102e8408774caf739a183403cfa705d799f5e77"} Jul 13 16:45:55.899191 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:55.899150 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 13 16:45:55.899447 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:55.899227 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 13 16:45:56.399634 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:56.399598 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" event={"ID":"ded7ff20-866f-43ec-9815-3a3712ffea91","Type":"ContainerStarted","Data":"dc466b3a3c24e4cd0bbbaef9a733d36d010136f014716f2810399c3af03b52c4"} Jul 13 16:45:56.399813 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:56.399763 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:45:56.401606 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:56.401581 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-cgp86 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 16:45:56.401705 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:56.401625 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" podUID="ded7ff20-866f-43ec-9815-3a3712ffea91" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 16:45:56.423853 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:56.423808 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" podStartSLOduration=1.8449973339999999 podStartE2EDuration="4.423794192s" totalImagesPullingTime="2.578796858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:56.421341883 +0000 UTC m=+371.211429817" watchObservedRunningTime="2026-07-13 16:45:56.423794192 +0000 UTC m=+371.213882126" Jul 13 16:45:57.404565 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:57.404530 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-cgp86 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 16:45:57.404952 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:45:57.404589 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" podUID="ded7ff20-866f-43ec-9815-3a3712ffea91" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 16:46:00.405438 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:46:00.405407 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cgp86" Jul 13 16:47:01.406693 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.406649 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj"] Jul 13 16:47:01.410310 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.410287 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.412865 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.412844 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 16:47:01.413886 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.413868 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 16:47:01.413948 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.413879 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q7rkl\"" Jul 13 16:47:01.418707 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.418682 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj"] Jul 13 16:47:01.497555 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.497513 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbj4\" (UniqueName: \"kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.497747 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.497566 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.497747 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.497653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.599012 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.598976 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.599189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.599072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbj4\" (UniqueName: \"kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.599189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.599100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.599421 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.599396 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.599469 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.599443 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.607725 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.607692 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbj4\" (UniqueName: \"kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.720131 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.720028 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:01.856070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:01.856029 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj"] Jul 13 16:47:01.857891 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:47:01.857860 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4728ae_9937_4dae_a2e7_14e89fd79f0d.slice/crio-53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1 WatchSource:0}: Error finding container 53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1: Status 404 returned error can't find the container with id 53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1 Jul 13 16:47:02.205025 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.204988 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr"] Jul 13 16:47:02.208502 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.208485 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.218324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.218299 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr"] Jul 13 16:47:02.304948 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.304918 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.305169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.304958 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.305169 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.304997 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdtwj\" (UniqueName: \"kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.405518 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.405486 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdtwj\" (UniqueName: \"kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.405700 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.405557 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.405700 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.405589 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.405915 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.405895 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.405966 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.405938 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.414067 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.414048 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdtwj\" (UniqueName: \"kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.517806 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.517725 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:02.668573 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.668512 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr"] Jul 13 16:47:02.671136 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:47:02.671114 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d4ea9b_2822_48ee_a769_55831ff12212.slice/crio-fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee WatchSource:0}: Error finding container fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee: Status 404 returned error can't find the container with id fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee Jul 13 16:47:02.713606 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.713571 2569 generic.go:356] "Generic (PLEG): container finished" podID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerID="afe462f802ae663393685790b01b81ebb89732e91185c51f82cec1b20226bdbd" exitCode=0 Jul 13 16:47:02.713733 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.713648 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" event={"ID":"af4728ae-9937-4dae-a2e7-14e89fd79f0d","Type":"ContainerDied","Data":"afe462f802ae663393685790b01b81ebb89732e91185c51f82cec1b20226bdbd"} Jul 13 16:47:02.713733 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.713709 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" event={"ID":"af4728ae-9937-4dae-a2e7-14e89fd79f0d","Type":"ContainerStarted","Data":"53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1"} Jul 13 16:47:02.715150 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.715119 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" event={"ID":"78d4ea9b-2822-48ee-a769-55831ff12212","Type":"ContainerStarted","Data":"fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee"} Jul 13 16:47:02.814096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.814062 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5"] Jul 13 16:47:02.817360 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.817345 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:02.825535 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.825511 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5"] Jul 13 16:47:02.909919 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.909886 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:02.910096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.909926 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v92wk\" (UniqueName: \"kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:02.910096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:02.910015 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.010433 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.010390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.010609 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.010446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.010609 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.010487 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v92wk\" (UniqueName: \"kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.010785 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.010763 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.010830 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.010812 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.026419 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.026391 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v92wk\" (UniqueName: \"kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.174824 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.174779 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:03.245264 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.245234 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc"] Jul 13 16:47:03.248870 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.248848 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.264594 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.264569 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc"] Jul 13 16:47:03.312876 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.312635 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm9tv\" (UniqueName: \"kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.312876 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.312703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.312876 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.312736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.315559 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.315530 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5"] Jul 13 16:47:03.318683 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:47:03.318658 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a086ea_fed8_4596_a55d_443d3b1381ca.slice/crio-a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6 WatchSource:0}: Error finding container a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6: Status 404 returned error can't find the container with id a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6 Jul 13 16:47:03.413956 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.413912 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm9tv\" (UniqueName: \"kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.413956 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.413955 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.414575 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.414005 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.414575 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.414303 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.414575 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.414355 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.426260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.426202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm9tv\" (UniqueName: \"kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.560082 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.560031 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:03.706367 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.706295 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc"] Jul 13 16:47:03.709067 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:47:03.709025 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928158e2_5a02_4549_beb5_bfb28e65a1a4.slice/crio-dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e WatchSource:0}: Error finding container dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e: Status 404 returned error can't find the container with id dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e Jul 13 16:47:03.722489 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.722463 2569 generic.go:356] "Generic (PLEG): container finished" podID="78d4ea9b-2822-48ee-a769-55831ff12212" containerID="8bda772513d55a3b4f474e02d47f41ec163eb367caa57ced62598b85330e3a94" exitCode=0 Jul 13 16:47:03.722604 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.722548 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" event={"ID":"78d4ea9b-2822-48ee-a769-55831ff12212","Type":"ContainerDied","Data":"8bda772513d55a3b4f474e02d47f41ec163eb367caa57ced62598b85330e3a94"} Jul 13 16:47:03.723792 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.723767 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" event={"ID":"928158e2-5a02-4549-beb5-bfb28e65a1a4","Type":"ContainerStarted","Data":"dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e"} Jul 13 16:47:03.725470 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.725445 2569 generic.go:356] "Generic (PLEG): container finished" podID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerID="c3ddc85784fc55dcd97ba9ea5caf6ea104841d663ba1495c30c1e8e4ef226690" exitCode=0 Jul 13 16:47:03.725572 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.725522 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" event={"ID":"80a086ea-fed8-4596-a55d-443d3b1381ca","Type":"ContainerDied","Data":"c3ddc85784fc55dcd97ba9ea5caf6ea104841d663ba1495c30c1e8e4ef226690"} Jul 13 16:47:03.725572 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:03.725551 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" event={"ID":"80a086ea-fed8-4596-a55d-443d3b1381ca","Type":"ContainerStarted","Data":"a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6"} Jul 13 16:47:04.731849 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.731818 2569 generic.go:356] "Generic (PLEG): container finished" podID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerID="eb97534529e95d3488df08f9efa391af6a1e7e5c30c790db6ddb52fca726806c" exitCode=0 Jul 13 16:47:04.732198 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.731896 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" event={"ID":"af4728ae-9937-4dae-a2e7-14e89fd79f0d","Type":"ContainerDied","Data":"eb97534529e95d3488df08f9efa391af6a1e7e5c30c790db6ddb52fca726806c"} Jul 13 16:47:04.733770 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.733750 2569 generic.go:356] "Generic (PLEG): container finished" podID="78d4ea9b-2822-48ee-a769-55831ff12212" containerID="cd39576b71916e627a94c54f020d5c692e9ec9d9c38d5d3b97b5a402ef62d073" exitCode=0 Jul 13 16:47:04.733865 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.733830 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" event={"ID":"78d4ea9b-2822-48ee-a769-55831ff12212","Type":"ContainerDied","Data":"cd39576b71916e627a94c54f020d5c692e9ec9d9c38d5d3b97b5a402ef62d073"} Jul 13 16:47:04.735492 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.735472 2569 generic.go:356] "Generic (PLEG): container finished" podID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerID="9a9e381368382b9ebf6d706b85a027b0980ac29b27f4dd922d43cdcf2a06f680" exitCode=0 Jul 13 16:47:04.735584 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.735538 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" event={"ID":"928158e2-5a02-4549-beb5-bfb28e65a1a4","Type":"ContainerDied","Data":"9a9e381368382b9ebf6d706b85a027b0980ac29b27f4dd922d43cdcf2a06f680"} Jul 13 16:47:04.737287 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.737270 2569 generic.go:356] "Generic (PLEG): container finished" podID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerID="fd68f4496a36161ddfa83757a343ed1cec26763c53ec4bbd0422d4d389fc4c19" exitCode=0 Jul 13 16:47:04.737385 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:04.737296 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" event={"ID":"80a086ea-fed8-4596-a55d-443d3b1381ca","Type":"ContainerDied","Data":"fd68f4496a36161ddfa83757a343ed1cec26763c53ec4bbd0422d4d389fc4c19"} Jul 13 16:47:05.743934 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.743902 2569 generic.go:356] "Generic (PLEG): container finished" podID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerID="26bc5bb4c2c6225631412f5ea73a6bc2910618b92bf91dccf77e537c062e9014" exitCode=0 Jul 13 16:47:05.744320 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.743985 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" event={"ID":"af4728ae-9937-4dae-a2e7-14e89fd79f0d","Type":"ContainerDied","Data":"26bc5bb4c2c6225631412f5ea73a6bc2910618b92bf91dccf77e537c062e9014"} Jul 13 16:47:05.746056 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.746021 2569 generic.go:356] "Generic (PLEG): container finished" podID="78d4ea9b-2822-48ee-a769-55831ff12212" containerID="53c5d27a95a44d374eedc0b7d2d239a24adac21b1b10929240e1c4fa0a083ac0" exitCode=0 Jul 13 16:47:05.746126 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.746073 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" event={"ID":"78d4ea9b-2822-48ee-a769-55831ff12212","Type":"ContainerDied","Data":"53c5d27a95a44d374eedc0b7d2d239a24adac21b1b10929240e1c4fa0a083ac0"} Jul 13 16:47:05.747788 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.747764 2569 generic.go:356] "Generic (PLEG): container finished" podID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerID="71d761ec781e775a136cc8a5f94b1e02a87bd015db22e3a10805101e2f17a1b5" exitCode=0 Jul 13 16:47:05.747904 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.747831 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" event={"ID":"928158e2-5a02-4549-beb5-bfb28e65a1a4","Type":"ContainerDied","Data":"71d761ec781e775a136cc8a5f94b1e02a87bd015db22e3a10805101e2f17a1b5"} Jul 13 16:47:05.750191 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.750145 2569 generic.go:356] "Generic (PLEG): container finished" podID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerID="2f73618ec77ca7d3b793023575ea3feaec6568b2dfa9dd3b6c9ef2ac1950ebcf" exitCode=0 Jul 13 16:47:05.750191 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:05.750188 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" event={"ID":"80a086ea-fed8-4596-a55d-443d3b1381ca","Type":"ContainerDied","Data":"2f73618ec77ca7d3b793023575ea3feaec6568b2dfa9dd3b6c9ef2ac1950ebcf"} Jul 13 16:47:06.756586 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:06.756547 2569 generic.go:356] "Generic (PLEG): container finished" podID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerID="b7da9f6f4722ff1feb4b3ef39198c1b6a92d376204fd1ed06e18879559cbc8d1" exitCode=0 Jul 13 16:47:06.757078 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:06.756633 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" event={"ID":"928158e2-5a02-4549-beb5-bfb28e65a1a4","Type":"ContainerDied","Data":"b7da9f6f4722ff1feb4b3ef39198c1b6a92d376204fd1ed06e18879559cbc8d1"} Jul 13 16:47:06.905583 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:06.905556 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:06.943878 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:06.943856 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:06.946761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:06.946738 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:07.044898 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.044819 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gbj4\" (UniqueName: \"kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4\") pod \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " Jul 13 16:47:07.044898 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.044863 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util\") pod \"78d4ea9b-2822-48ee-a769-55831ff12212\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " Jul 13 16:47:07.044898 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.044893 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle\") pod \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.044925 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle\") pod \"78d4ea9b-2822-48ee-a769-55831ff12212\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.044979 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util\") pod \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\" (UID: \"af4728ae-9937-4dae-a2e7-14e89fd79f0d\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045005 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle\") pod \"80a086ea-fed8-4596-a55d-443d3b1381ca\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045074 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v92wk\" (UniqueName: \"kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk\") pod \"80a086ea-fed8-4596-a55d-443d3b1381ca\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045116 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdtwj\" (UniqueName: \"kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj\") pod \"78d4ea9b-2822-48ee-a769-55831ff12212\" (UID: \"78d4ea9b-2822-48ee-a769-55831ff12212\") " Jul 13 16:47:07.045166 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045142 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util\") pod \"80a086ea-fed8-4596-a55d-443d3b1381ca\" (UID: \"80a086ea-fed8-4596-a55d-443d3b1381ca\") " Jul 13 16:47:07.045788 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045733 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle" (OuterVolumeSpecName: "bundle") pod "80a086ea-fed8-4596-a55d-443d3b1381ca" (UID: "80a086ea-fed8-4596-a55d-443d3b1381ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.045909 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.045818 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle" (OuterVolumeSpecName: "bundle") pod "78d4ea9b-2822-48ee-a769-55831ff12212" (UID: "78d4ea9b-2822-48ee-a769-55831ff12212"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.046051 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.046016 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle" (OuterVolumeSpecName: "bundle") pod "af4728ae-9937-4dae-a2e7-14e89fd79f0d" (UID: "af4728ae-9937-4dae-a2e7-14e89fd79f0d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.047599 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.047571 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4" (OuterVolumeSpecName: "kube-api-access-5gbj4") pod "af4728ae-9937-4dae-a2e7-14e89fd79f0d" (UID: "af4728ae-9937-4dae-a2e7-14e89fd79f0d"). InnerVolumeSpecName "kube-api-access-5gbj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:47:07.047718 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.047672 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk" (OuterVolumeSpecName: "kube-api-access-v92wk") pod "80a086ea-fed8-4596-a55d-443d3b1381ca" (UID: "80a086ea-fed8-4596-a55d-443d3b1381ca"). InnerVolumeSpecName "kube-api-access-v92wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:47:07.047901 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.047873 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj" (OuterVolumeSpecName: "kube-api-access-bdtwj") pod "78d4ea9b-2822-48ee-a769-55831ff12212" (UID: "78d4ea9b-2822-48ee-a769-55831ff12212"). InnerVolumeSpecName "kube-api-access-bdtwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:47:07.051085 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.051065 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util" (OuterVolumeSpecName: "util") pod "af4728ae-9937-4dae-a2e7-14e89fd79f0d" (UID: "af4728ae-9937-4dae-a2e7-14e89fd79f0d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.053511 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.053489 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util" (OuterVolumeSpecName: "util") pod "80a086ea-fed8-4596-a55d-443d3b1381ca" (UID: "80a086ea-fed8-4596-a55d-443d3b1381ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.053866 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.053850 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util" (OuterVolumeSpecName: "util") pod "78d4ea9b-2822-48ee-a769-55831ff12212" (UID: "78d4ea9b-2822-48ee-a769-55831ff12212"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:07.146324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146289 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146321 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146330 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78d4ea9b-2822-48ee-a769-55831ff12212-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146324 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146337 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4728ae-9937-4dae-a2e7-14e89fd79f0d-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146346 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146355 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v92wk\" (UniqueName: \"kubernetes.io/projected/80a086ea-fed8-4596-a55d-443d3b1381ca-kube-api-access-v92wk\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146364 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdtwj\" (UniqueName: \"kubernetes.io/projected/78d4ea9b-2822-48ee-a769-55831ff12212-kube-api-access-bdtwj\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146371 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a086ea-fed8-4596-a55d-443d3b1381ca-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.146379 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gbj4\" (UniqueName: \"kubernetes.io/projected/af4728ae-9937-4dae-a2e7-14e89fd79f0d-kube-api-access-5gbj4\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:07.762937 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.762908 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" Jul 13 16:47:07.763312 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.762905 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5" event={"ID":"80a086ea-fed8-4596-a55d-443d3b1381ca","Type":"ContainerDied","Data":"a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6"} Jul 13 16:47:07.763312 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.763022 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a893461e9ed8d82fb797c5a60495c997d64170920fe26bc33f30afb5e27fc5e6" Jul 13 16:47:07.764947 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.764930 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" Jul 13 16:47:07.765159 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.764961 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj" event={"ID":"af4728ae-9937-4dae-a2e7-14e89fd79f0d","Type":"ContainerDied","Data":"53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1"} Jul 13 16:47:07.765159 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.765003 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53862673c38180b743b6ca360441b4cdb84f6554c750a5e621e0eac45e7365c1" Jul 13 16:47:07.766951 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.766902 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" event={"ID":"78d4ea9b-2822-48ee-a769-55831ff12212","Type":"ContainerDied","Data":"fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee"} Jul 13 16:47:07.767062 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.766954 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr" Jul 13 16:47:07.767062 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.766970 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbb7dc8da6d95dcb0142cc856625af8afe49a3f4ca83f2a28c69cde68e7981ee" Jul 13 16:47:07.882872 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:07.882847 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:08.054138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.054058 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm9tv\" (UniqueName: \"kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv\") pod \"928158e2-5a02-4549-beb5-bfb28e65a1a4\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " Jul 13 16:47:08.054138 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.054115 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util\") pod \"928158e2-5a02-4549-beb5-bfb28e65a1a4\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " Jul 13 16:47:08.054337 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.054175 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle\") pod \"928158e2-5a02-4549-beb5-bfb28e65a1a4\" (UID: \"928158e2-5a02-4549-beb5-bfb28e65a1a4\") " Jul 13 16:47:08.054638 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.054608 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle" (OuterVolumeSpecName: "bundle") pod "928158e2-5a02-4549-beb5-bfb28e65a1a4" (UID: "928158e2-5a02-4549-beb5-bfb28e65a1a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:08.056053 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.056013 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv" (OuterVolumeSpecName: "kube-api-access-cm9tv") pod "928158e2-5a02-4549-beb5-bfb28e65a1a4" (UID: "928158e2-5a02-4549-beb5-bfb28e65a1a4"). InnerVolumeSpecName "kube-api-access-cm9tv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:47:08.059712 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.059674 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util" (OuterVolumeSpecName: "util") pod "928158e2-5a02-4549-beb5-bfb28e65a1a4" (UID: "928158e2-5a02-4549-beb5-bfb28e65a1a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 16:47:08.154954 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.154888 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm9tv\" (UniqueName: \"kubernetes.io/projected/928158e2-5a02-4549-beb5-bfb28e65a1a4-kube-api-access-cm9tv\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:08.154954 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.154912 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-util\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:08.154954 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.154921 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/928158e2-5a02-4549-beb5-bfb28e65a1a4-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:47:08.774691 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.774661 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" event={"ID":"928158e2-5a02-4549-beb5-bfb28e65a1a4","Type":"ContainerDied","Data":"dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e"} Jul 13 16:47:08.774691 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.774689 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc" Jul 13 16:47:08.775120 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:47:08.774705 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfa09b3ab8459ef531a415e3a277a5fce9dcee953e2becf4f839e9bf8046265e" Jul 13 16:48:04.227789 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:04.227743 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:48:15.139728 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.139686 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:15.140218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140165 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="pull" Jul 13 16:48:15.140218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140183 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="pull" Jul 13 16:48:15.140218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140196 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="pull" Jul 13 16:48:15.140218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140204 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="pull" Jul 13 16:48:15.140218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140216 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140225 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140235 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140242 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140250 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140258 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140269 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140277 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140290 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140297 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140311 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="pull" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140319 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="pull" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140333 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140340 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140356 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="pull" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140364 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="pull" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140377 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140384 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="util" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140395 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="extract" Jul 13 16:48:15.140477 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140403 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="extract" Jul 13 16:48:15.141395 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140491 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="78d4ea9b-2822-48ee-a769-55831ff12212" containerName="extract" Jul 13 16:48:15.141395 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140505 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="928158e2-5a02-4549-beb5-bfb28e65a1a4" containerName="extract" Jul 13 16:48:15.141395 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140517 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4728ae-9937-4dae-a2e7-14e89fd79f0d" containerName="extract" Jul 13 16:48:15.141395 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.140528 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a086ea-fed8-4596-a55d-443d3b1381ca" containerName="extract" Jul 13 16:48:15.143757 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.143736 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.146474 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.146446 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 13 16:48:15.146955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.146938 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 16:48:15.147098 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.147079 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 16:48:15.147654 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.147639 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q7rkl\"" Jul 13 16:48:15.157822 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.157799 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:15.233918 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.233882 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:15.295823 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.295792 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.295982 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.295847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppl2h\" (UniqueName: \"kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.396506 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.396480 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.396669 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.396539 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppl2h\" (UniqueName: \"kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.397107 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.397090 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.413061 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.413015 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppl2h\" (UniqueName: \"kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h\") pod \"limitador-limitador-64c94fd64f-cd9d4\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.453669 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.453640 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:15.593939 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:15.593913 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:15.595855 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:48:15.595832 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78dfb574_9cfe_4466_9ec0_79ea91d4eeee.slice/crio-20ffdecd65dbf4a6431582dcd3efec4f50bd7a544f5dfc4a438b20dda208c49a WatchSource:0}: Error finding container 20ffdecd65dbf4a6431582dcd3efec4f50bd7a544f5dfc4a438b20dda208c49a: Status 404 returned error can't find the container with id 20ffdecd65dbf4a6431582dcd3efec4f50bd7a544f5dfc4a438b20dda208c49a Jul 13 16:48:16.100207 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:16.100172 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" event={"ID":"78dfb574-9cfe-4466-9ec0-79ea91d4eeee","Type":"ContainerStarted","Data":"20ffdecd65dbf4a6431582dcd3efec4f50bd7a544f5dfc4a438b20dda208c49a"} Jul 13 16:48:19.118063 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:19.118010 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" event={"ID":"78dfb574-9cfe-4466-9ec0-79ea91d4eeee","Type":"ContainerStarted","Data":"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2"} Jul 13 16:48:19.118424 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:19.118209 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:19.145244 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:19.145187 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" podStartSLOduration=1.434584608 podStartE2EDuration="4.145171668s" totalImagesPullingTime="2.71058706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:48:19.142431741 +0000 UTC m=+513.932519676" watchObservedRunningTime="2026-07-13 16:48:19.145171668 +0000 UTC m=+513.935259601" Jul 13 16:48:29.247955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.247892 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f64ccdf7-jx2hn" podUID="26458d20-2395-4cab-975f-8d531d83374a" containerName="console" containerID="cri-o://a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44" gracePeriod=15 Jul 13 16:48:29.499303 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.499247 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:48:29.518515 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518490 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxxxv\" (UniqueName: \"kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.518674 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518545 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.518674 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518599 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.518674 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518626 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.518674 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518653 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.518875 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518690 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.519076 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.519030 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config" (OuterVolumeSpecName: "console-config") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:48:29.519148 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.518957 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca" (OuterVolumeSpecName: "service-ca") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:48:29.519205 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.519173 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:48:29.521611 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.521567 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:48:29.521727 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.521607 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv" (OuterVolumeSpecName: "kube-api-access-nxxxv") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "kube-api-access-nxxxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:48:29.522096 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.522015 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:48:29.619285 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619241 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert\") pod \"26458d20-2395-4cab-975f-8d531d83374a\" (UID: \"26458d20-2395-4cab-975f-8d531d83374a\") " Jul 13 16:48:29.619441 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619428 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxxxv\" (UniqueName: \"kubernetes.io/projected/26458d20-2395-4cab-975f-8d531d83374a-kube-api-access-nxxxv\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619446 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619457 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-oauth-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619466 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-console-config\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619474 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26458d20-2395-4cab-975f-8d531d83374a-console-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619482 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-trusted-ca-bundle\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:29.619726 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.619644 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26458d20-2395-4cab-975f-8d531d83374a" (UID: "26458d20-2395-4cab-975f-8d531d83374a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:48:29.719954 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:29.719927 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26458d20-2395-4cab-975f-8d531d83374a-oauth-serving-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:30.124258 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.124226 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:30.174844 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.174814 2569 generic.go:356] "Generic (PLEG): container finished" podID="26458d20-2395-4cab-975f-8d531d83374a" containerID="a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44" exitCode=0 Jul 13 16:48:30.175010 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.174857 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f64ccdf7-jx2hn" Jul 13 16:48:30.175010 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.174902 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f64ccdf7-jx2hn" event={"ID":"26458d20-2395-4cab-975f-8d531d83374a","Type":"ContainerDied","Data":"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44"} Jul 13 16:48:30.175010 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.174953 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f64ccdf7-jx2hn" event={"ID":"26458d20-2395-4cab-975f-8d531d83374a","Type":"ContainerDied","Data":"be73955777cd4d7d7bec0e4e616bdf0b2e312f27c6425e8d0d9bcd8bb6187f2d"} Jul 13 16:48:30.175010 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.174979 2569 scope.go:117] "RemoveContainer" containerID="a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44" Jul 13 16:48:30.186881 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.186863 2569 scope.go:117] "RemoveContainer" containerID="a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44" Jul 13 16:48:30.187160 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:48:30.187137 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44\": container with ID starting with a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44 not found: ID does not exist" containerID="a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44" Jul 13 16:48:30.187245 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.187167 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44"} err="failed to get container status \"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44\": rpc error: code = NotFound desc = could not find container \"a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44\": container with ID starting with a0913aec1f1bfef3c5071eebe20ee68c730f954a4d7ba86167191491a0702e44 not found: ID does not exist" Jul 13 16:48:30.194157 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.194134 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:48:30.198114 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:30.198092 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f64ccdf7-jx2hn"] Jul 13 16:48:31.297015 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.296980 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:31.297427 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.297198 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" podUID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" containerName="limitador" containerID="cri-o://2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2" gracePeriod=30 Jul 13 16:48:31.720186 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.720155 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26458d20-2395-4cab-975f-8d531d83374a" path="/var/lib/kubelet/pods/26458d20-2395-4cab-975f-8d531d83374a/volumes" Jul 13 16:48:31.846665 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.846640 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:31.932794 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.932770 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file\") pod \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " Jul 13 16:48:31.932937 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.932812 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppl2h\" (UniqueName: \"kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h\") pod \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\" (UID: \"78dfb574-9cfe-4466-9ec0-79ea91d4eeee\") " Jul 13 16:48:31.933142 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.933120 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file" (OuterVolumeSpecName: "config-file") pod "78dfb574-9cfe-4466-9ec0-79ea91d4eeee" (UID: "78dfb574-9cfe-4466-9ec0-79ea91d4eeee"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 16:48:31.934778 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:31.934752 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h" (OuterVolumeSpecName: "kube-api-access-ppl2h") pod "78dfb574-9cfe-4466-9ec0-79ea91d4eeee" (UID: "78dfb574-9cfe-4466-9ec0-79ea91d4eeee"). InnerVolumeSpecName "kube-api-access-ppl2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:48:32.033398 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.033363 2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-config-file\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:32.033398 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.033389 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppl2h\" (UniqueName: \"kubernetes.io/projected/78dfb574-9cfe-4466-9ec0-79ea91d4eeee-kube-api-access-ppl2h\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:48:32.187279 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.187198 2569 generic.go:356] "Generic (PLEG): container finished" podID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" containerID="2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2" exitCode=0 Jul 13 16:48:32.187279 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.187240 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" Jul 13 16:48:32.187279 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.187237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" event={"ID":"78dfb574-9cfe-4466-9ec0-79ea91d4eeee","Type":"ContainerDied","Data":"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2"} Jul 13 16:48:32.187513 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.187301 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cd9d4" event={"ID":"78dfb574-9cfe-4466-9ec0-79ea91d4eeee","Type":"ContainerDied","Data":"20ffdecd65dbf4a6431582dcd3efec4f50bd7a544f5dfc4a438b20dda208c49a"} Jul 13 16:48:32.187513 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.187334 2569 scope.go:117] "RemoveContainer" containerID="2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2" Jul 13 16:48:32.196153 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.196134 2569 scope.go:117] "RemoveContainer" containerID="2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2" Jul 13 16:48:32.196403 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:48:32.196384 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2\": container with ID starting with 2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2 not found: ID does not exist" containerID="2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2" Jul 13 16:48:32.196469 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.196415 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2"} err="failed to get container status \"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2\": rpc error: code = NotFound desc = could not find container \"2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2\": container with ID starting with 2d0ad51331175ffba7fc7138cda8c41ff6be6fa9e78eafbb227f67184895b4e2 not found: ID does not exist" Jul 13 16:48:32.209530 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.209508 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:32.217732 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:32.217711 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cd9d4"] Jul 13 16:48:33.719921 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:33.719880 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" path="/var/lib/kubelet/pods/78dfb574-9cfe-4466-9ec0-79ea91d4eeee/volumes" Jul 13 16:48:37.117968 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.117935 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zddt"] Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118261 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" containerName="limitador" Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118273 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" containerName="limitador" Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118286 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26458d20-2395-4cab-975f-8d531d83374a" containerName="console" Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118291 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="26458d20-2395-4cab-975f-8d531d83374a" containerName="console" Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118347 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="78dfb574-9cfe-4466-9ec0-79ea91d4eeee" containerName="limitador" Jul 13 16:48:37.118481 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.118358 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="26458d20-2395-4cab-975f-8d531d83374a" containerName="console" Jul 13 16:48:37.124177 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.124154 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.131183 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.131165 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cf876\"" Jul 13 16:48:37.131285 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.131246 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 16:48:37.132299 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.132284 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 16:48:37.133866 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.133852 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 13 16:48:37.147453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.147427 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zddt"] Jul 13 16:48:37.173202 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.173170 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5d7f546-740d-4791-b7a7-a60f8f452e21-data\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.173306 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.173208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnl7g\" (UniqueName: \"kubernetes.io/projected/f5d7f546-740d-4791-b7a7-a60f8f452e21-kube-api-access-mnl7g\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.274343 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.274315 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5d7f546-740d-4791-b7a7-a60f8f452e21-data\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.274488 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.274350 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnl7g\" (UniqueName: \"kubernetes.io/projected/f5d7f546-740d-4791-b7a7-a60f8f452e21-kube-api-access-mnl7g\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.274703 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.274683 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5d7f546-740d-4791-b7a7-a60f8f452e21-data\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.283760 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.283734 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnl7g\" (UniqueName: \"kubernetes.io/projected/f5d7f546-740d-4791-b7a7-a60f8f452e21-kube-api-access-mnl7g\") pod \"postgres-69fb4594cf-4zddt\" (UID: \"f5d7f546-740d-4791-b7a7-a60f8f452e21\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.433802 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.433764 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:37.566410 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:37.566382 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4zddt"] Jul 13 16:48:37.568468 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:48:37.568436 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d7f546_740d_4791_b7a7_a60f8f452e21.slice/crio-fbd2bc1205a875b555c11010b3d62743a0b9ac1f0f4f812860350e1b0bd6936b WatchSource:0}: Error finding container fbd2bc1205a875b555c11010b3d62743a0b9ac1f0f4f812860350e1b0bd6936b: Status 404 returned error can't find the container with id fbd2bc1205a875b555c11010b3d62743a0b9ac1f0f4f812860350e1b0bd6936b Jul 13 16:48:38.216902 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:38.216822 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" event={"ID":"f5d7f546-740d-4791-b7a7-a60f8f452e21","Type":"ContainerStarted","Data":"fbd2bc1205a875b555c11010b3d62743a0b9ac1f0f4f812860350e1b0bd6936b"} Jul 13 16:48:43.244937 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:43.244906 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" event={"ID":"f5d7f546-740d-4791-b7a7-a60f8f452e21","Type":"ContainerStarted","Data":"7f5b710d788fd448b9c465f1229bd2e5dec701f940ec95e38626afc948d93525"} Jul 13 16:48:43.245347 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:43.244962 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:43.268219 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:43.268170 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" podStartSLOduration=1.373131795 podStartE2EDuration="6.268155901s" totalImagesPullingTime="4.895024106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:48:43.266963742 +0000 UTC m=+538.057051689" watchObservedRunningTime="2026-07-13 16:48:43.268155901 +0000 UTC m=+538.058243879" Jul 13 16:48:49.279955 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:49.279923 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4zddt" Jul 13 16:48:57.610420 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.610383 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:48:57.614980 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.614961 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.617954 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.617934 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tqpmc\"" Jul 13 16:48:57.618074 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.617933 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 16:48:57.619007 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.618989 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 16:48:57.625444 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.625424 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:48:57.746023 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.745992 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkzdv\" (UniqueName: \"kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.746203 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.746034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.838128 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.838096 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:48:57.841433 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.841407 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:57.846747 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.846720 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkzdv\" (UniqueName: \"kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.846897 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.846765 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.849471 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.849448 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.856137 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.856115 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:48:57.861486 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.861436 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkzdv\" (UniqueName: \"kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv\") pod \"maas-controller-6574dbb6b5-8rxmq\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.926030 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.925999 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:48:57.948150 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.948114 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:57.948270 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:57.948198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqb42\" (UniqueName: \"kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.049349 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.049314 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.049523 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.049371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqb42\" (UniqueName: \"kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.051644 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.051619 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.064412 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.064385 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:48:58.067080 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:48:58.067059 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bef590_b845_4975_bbde_720340a402fe.slice/crio-0842976198c952ee1085f1a2c4afdd1495bdee87a9ea4ee36623a3d70eefa3aa WatchSource:0}: Error finding container 0842976198c952ee1085f1a2c4afdd1495bdee87a9ea4ee36623a3d70eefa3aa: Status 404 returned error can't find the container with id 0842976198c952ee1085f1a2c4afdd1495bdee87a9ea4ee36623a3d70eefa3aa Jul 13 16:48:58.069229 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.069197 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqb42\" (UniqueName: \"kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42\") pod \"maas-controller-69cbf74464-6q4rt\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.151372 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.151346 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:48:58.282196 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.282173 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:48:58.283815 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:48:58.283786 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc6b2b5_c8d2_42de_a39f_f87932ee644a.slice/crio-a83a0c7e5e6b71313f5e78d7efea25b157180391fecffb0c06efcfa0774fa68f WatchSource:0}: Error finding container a83a0c7e5e6b71313f5e78d7efea25b157180391fecffb0c06efcfa0774fa68f: Status 404 returned error can't find the container with id a83a0c7e5e6b71313f5e78d7efea25b157180391fecffb0c06efcfa0774fa68f Jul 13 16:48:58.327847 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.327822 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-6q4rt" event={"ID":"3fc6b2b5-c8d2-42de-a39f-f87932ee644a","Type":"ContainerStarted","Data":"a83a0c7e5e6b71313f5e78d7efea25b157180391fecffb0c06efcfa0774fa68f"} Jul 13 16:48:58.328904 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:48:58.328874 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" event={"ID":"94bef590-b845-4975-bbde-720340a402fe","Type":"ContainerStarted","Data":"0842976198c952ee1085f1a2c4afdd1495bdee87a9ea4ee36623a3d70eefa3aa"} Jul 13 16:49:01.348831 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.348786 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-6q4rt" event={"ID":"3fc6b2b5-c8d2-42de-a39f-f87932ee644a","Type":"ContainerStarted","Data":"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176"} Jul 13 16:49:01.349308 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.348900 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:49:01.350340 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.350317 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" event={"ID":"94bef590-b845-4975-bbde-720340a402fe","Type":"ContainerStarted","Data":"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d"} Jul 13 16:49:01.350446 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.350400 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:49:01.369227 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.369180 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69cbf74464-6q4rt" podStartSLOduration=2.291110921 podStartE2EDuration="4.369166229s" totalImagesPullingTime="2.078055308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:49:01.367778653 +0000 UTC m=+556.157866611" watchObservedRunningTime="2026-07-13 16:49:01.369166229 +0000 UTC m=+556.159254163" Jul 13 16:49:01.389195 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:01.389147 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" podStartSLOduration=2.083559137 podStartE2EDuration="4.389132405s" totalImagesPullingTime="2.305573268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:49:01.387034011 +0000 UTC m=+556.177121967" watchObservedRunningTime="2026-07-13 16:49:01.389132405 +0000 UTC m=+556.179220340" Jul 13 16:49:12.357023 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.356987 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:49:12.357436 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.357115 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:49:12.431289 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.431257 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:49:12.431480 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.431431 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" podUID="94bef590-b845-4975-bbde-720340a402fe" containerName="manager" containerID="cri-o://708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d" gracePeriod=30 Jul 13 16:49:12.678542 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.678514 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:49:12.771846 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.771818 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkzdv\" (UniqueName: \"kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv\") pod \"94bef590-b845-4975-bbde-720340a402fe\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " Jul 13 16:49:12.772004 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.771872 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert\") pod \"94bef590-b845-4975-bbde-720340a402fe\" (UID: \"94bef590-b845-4975-bbde-720340a402fe\") " Jul 13 16:49:12.773984 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.773957 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "94bef590-b845-4975-bbde-720340a402fe" (UID: "94bef590-b845-4975-bbde-720340a402fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:49:12.774121 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.774088 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv" (OuterVolumeSpecName: "kube-api-access-rkzdv") pod "94bef590-b845-4975-bbde-720340a402fe" (UID: "94bef590-b845-4975-bbde-720340a402fe"). InnerVolumeSpecName "kube-api-access-rkzdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:49:12.872841 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.872804 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkzdv\" (UniqueName: \"kubernetes.io/projected/94bef590-b845-4975-bbde-720340a402fe-kube-api-access-rkzdv\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:49:12.872841 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:12.872833 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94bef590-b845-4975-bbde-720340a402fe-webhook-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:49:13.410105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.410067 2569 generic.go:356] "Generic (PLEG): container finished" podID="94bef590-b845-4975-bbde-720340a402fe" containerID="708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d" exitCode=0 Jul 13 16:49:13.410105 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.410106 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" Jul 13 16:49:13.410664 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.410145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" event={"ID":"94bef590-b845-4975-bbde-720340a402fe","Type":"ContainerDied","Data":"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d"} Jul 13 16:49:13.410664 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.410203 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6574dbb6b5-8rxmq" event={"ID":"94bef590-b845-4975-bbde-720340a402fe","Type":"ContainerDied","Data":"0842976198c952ee1085f1a2c4afdd1495bdee87a9ea4ee36623a3d70eefa3aa"} Jul 13 16:49:13.410664 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.410225 2569 scope.go:117] "RemoveContainer" containerID="708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d" Jul 13 16:49:13.419575 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.419558 2569 scope.go:117] "RemoveContainer" containerID="708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d" Jul 13 16:49:13.419860 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:49:13.419842 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d\": container with ID starting with 708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d not found: ID does not exist" containerID="708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d" Jul 13 16:49:13.419921 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.419874 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d"} err="failed to get container status \"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d\": rpc error: code = NotFound desc = could not find container \"708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d\": container with ID starting with 708422bd0601a4965ae297a3c3147056bddb6585bcc57afda0845c558634df4d not found: ID does not exist" Jul 13 16:49:13.433410 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.433385 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:49:13.439131 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.439109 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6574dbb6b5-8rxmq"] Jul 13 16:49:13.720501 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:13.720420 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94bef590-b845-4975-bbde-720340a402fe" path="/var/lib/kubelet/pods/94bef590-b845-4975-bbde-720340a402fe/volumes" Jul 13 16:49:34.779713 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.779677 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t"] Jul 13 16:49:34.780189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.780079 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94bef590-b845-4975-bbde-720340a402fe" containerName="manager" Jul 13 16:49:34.780189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.780092 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="94bef590-b845-4975-bbde-720340a402fe" containerName="manager" Jul 13 16:49:34.780189 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.780160 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="94bef590-b845-4975-bbde-720340a402fe" containerName="manager" Jul 13 16:49:34.803495 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.803472 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t"] Jul 13 16:49:34.803637 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.803563 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.806450 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.806425 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zphj7\"" Jul 13 16:49:34.806601 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.806584 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 16:49:34.847154 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.847120 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1838a6a2-a23b-4059-8f96-3bff84ca679b-maas-api-tls\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.847320 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.847204 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1838a6a2-a23b-4059-8f96-3bff84ca679b-openshift-service-ca\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.847320 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.847265 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqnwm\" (UniqueName: \"kubernetes.io/projected/1838a6a2-a23b-4059-8f96-3bff84ca679b-kube-api-access-lqnwm\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.948099 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.948065 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1838a6a2-a23b-4059-8f96-3bff84ca679b-maas-api-tls\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.948257 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.948118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1838a6a2-a23b-4059-8f96-3bff84ca679b-openshift-service-ca\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.948257 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.948138 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqnwm\" (UniqueName: \"kubernetes.io/projected/1838a6a2-a23b-4059-8f96-3bff84ca679b-kube-api-access-lqnwm\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.948752 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.948728 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1838a6a2-a23b-4059-8f96-3bff84ca679b-openshift-service-ca\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.950345 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.950326 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1838a6a2-a23b-4059-8f96-3bff84ca679b-maas-api-tls\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:34.957291 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:34.957269 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqnwm\" (UniqueName: \"kubernetes.io/projected/1838a6a2-a23b-4059-8f96-3bff84ca679b-kube-api-access-lqnwm\") pod \"maas-api-6ddc58579c-4n76t\" (UID: \"1838a6a2-a23b-4059-8f96-3bff84ca679b\") " pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:35.114363 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.114276 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:35.258210 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.258180 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t"] Jul 13 16:49:35.261349 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:49:35.261323 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1838a6a2_a23b_4059_8f96_3bff84ca679b.slice/crio-f454fd8b1130efc880b34ac6d38e4e305c81d29241f0247800885a318c9c2e2e WatchSource:0}: Error finding container f454fd8b1130efc880b34ac6d38e4e305c81d29241f0247800885a318c9c2e2e: Status 404 returned error can't find the container with id f454fd8b1130efc880b34ac6d38e4e305c81d29241f0247800885a318c9c2e2e Jul 13 16:49:35.417704 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.417674 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-52brd"] Jul 13 16:49:35.422452 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.422430 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.425204 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.425185 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-x5cpm\"" Jul 13 16:49:35.425510 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.425496 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 16:49:35.450878 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.450850 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-52brd"] Jul 13 16:49:35.451229 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.451206 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dc2ce4-525b-4449-8dbf-a2a7b861c540-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.451294 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.451248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhm7d\" (UniqueName: \"kubernetes.io/projected/86dc2ce4-525b-4449-8dbf-a2a7b861c540-kube-api-access-hhm7d\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.521416 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.521386 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" event={"ID":"1838a6a2-a23b-4059-8f96-3bff84ca679b","Type":"ContainerStarted","Data":"f454fd8b1130efc880b34ac6d38e4e305c81d29241f0247800885a318c9c2e2e"} Jul 13 16:49:35.551625 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.551592 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dc2ce4-525b-4449-8dbf-a2a7b861c540-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.551790 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.551639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhm7d\" (UniqueName: \"kubernetes.io/projected/86dc2ce4-525b-4449-8dbf-a2a7b861c540-kube-api-access-hhm7d\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.552369 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.552349 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86dc2ce4-525b-4449-8dbf-a2a7b861c540-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.566174 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.566146 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhm7d\" (UniqueName: \"kubernetes.io/projected/86dc2ce4-525b-4449-8dbf-a2a7b861c540-kube-api-access-hhm7d\") pod \"payload-processing-f9b8d9b9b-52brd\" (UID: \"86dc2ce4-525b-4449-8dbf-a2a7b861c540\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.732740 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.732661 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:35.869442 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:35.869413 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-52brd"] Jul 13 16:49:35.871305 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:49:35.871276 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86dc2ce4_525b_4449_8dbf_a2a7b861c540.slice/crio-579fb4b3e2cd6b46e9a4b6db028db9c4a3e2569b7fdb46cd85a0cbf6d3fb57a0 WatchSource:0}: Error finding container 579fb4b3e2cd6b46e9a4b6db028db9c4a3e2569b7fdb46cd85a0cbf6d3fb57a0: Status 404 returned error can't find the container with id 579fb4b3e2cd6b46e9a4b6db028db9c4a3e2569b7fdb46cd85a0cbf6d3fb57a0 Jul 13 16:49:36.529649 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:36.529598 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" event={"ID":"86dc2ce4-525b-4449-8dbf-a2a7b861c540","Type":"ContainerStarted","Data":"579fb4b3e2cd6b46e9a4b6db028db9c4a3e2569b7fdb46cd85a0cbf6d3fb57a0"} Jul 13 16:49:37.537466 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.537383 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" event={"ID":"1838a6a2-a23b-4059-8f96-3bff84ca679b","Type":"ContainerStarted","Data":"8ee94ba7389a82eef6cc1b7cb2917f71f52977028cf400cfe957688c11531ff6"} Jul 13 16:49:37.537917 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.537498 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:37.539157 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.539122 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" event={"ID":"86dc2ce4-525b-4449-8dbf-a2a7b861c540","Type":"ContainerStarted","Data":"216f7fe98428e5f7cd770d61013d7e4469bed0fa7e9f60b653e7d920fa9bcad8"} Jul 13 16:49:37.539271 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.539253 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:49:37.556466 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.556418 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" podStartSLOduration=2.1060391960000002 podStartE2EDuration="3.556403764s" totalImagesPullingTime="1.450364568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:49:37.556094034 +0000 UTC m=+592.346181968" watchObservedRunningTime="2026-07-13 16:49:37.556403764 +0000 UTC m=+592.346491698" Jul 13 16:49:37.575055 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:37.574991 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" podStartSLOduration=1.232737459 podStartE2EDuration="2.574977804s" totalImagesPullingTime="1.342240345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:49:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:49:37.572514389 +0000 UTC m=+592.362602336" watchObservedRunningTime="2026-07-13 16:49:37.574977804 +0000 UTC m=+592.365065736" Jul 13 16:49:43.550221 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:43.550187 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6ddc58579c-4n76t" Jul 13 16:49:45.690032 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:45.690005 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:49:45.693824 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:45.693799 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:49:45.696260 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:45.696239 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:49:45.699914 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:45.699888 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:49:48.546868 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:49:48.546833 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-52brd" Jul 13 16:50:20.310446 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.310405 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5"] Jul 13 16:50:20.314737 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.314718 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.317811 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.317787 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 16:50:20.317916 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.317787 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 16:50:20.318875 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.318853 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xms2v\"" Jul 13 16:50:20.318977 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.318891 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 16:50:20.326897 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.326871 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5"] Jul 13 16:50:20.422302 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422268 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.422302 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422304 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.422514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422323 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.422514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422373 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xcxg\" (UniqueName: \"kubernetes.io/projected/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-kube-api-access-6xcxg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.422514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422397 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.422514 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.422424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523588 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523595 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xcxg\" (UniqueName: \"kubernetes.io/projected/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-kube-api-access-6xcxg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523658 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523761 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523701 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.523973 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.523794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.524116 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.524091 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.524116 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.524108 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.524240 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.524149 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.525804 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.525779 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.526144 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.526127 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.532033 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.532017 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xcxg\" (UniqueName: \"kubernetes.io/projected/b4d6ea6e-8bfc-44c1-9d4f-c277612870b2-kube-api-access-6xcxg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ht6w5\" (UID: \"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.626603 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.626524 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:20.761100 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.761072 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5"] Jul 13 16:50:20.763149 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:50:20.763121 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d6ea6e_8bfc_44c1_9d4f_c277612870b2.slice/crio-46dc8082067c3360fba952926e2d1c45e310817185dc3ce348be23e2132bb060 WatchSource:0}: Error finding container 46dc8082067c3360fba952926e2d1c45e310817185dc3ce348be23e2132bb060: Status 404 returned error can't find the container with id 46dc8082067c3360fba952926e2d1c45e310817185dc3ce348be23e2132bb060 Jul 13 16:50:20.764413 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:20.764394 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 16:50:21.764637 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:21.764599 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" event={"ID":"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2","Type":"ContainerStarted","Data":"46dc8082067c3360fba952926e2d1c45e310817185dc3ce348be23e2132bb060"} Jul 13 16:50:22.772106 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:22.772069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" event={"ID":"b4d6ea6e-8bfc-44c1-9d4f-c277612870b2","Type":"ContainerStarted","Data":"4d9de6b436af0baaeef2eb4d24be89753367ed1abe87dd72bb7603304a54b23d"} Jul 13 16:50:30.626737 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:30.626695 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:30.627127 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:30.626770 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:30.639347 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:30.639325 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:30.662534 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:30.662473 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" podStartSLOduration=9.636817252 podStartE2EDuration="10.662450873s" totalImagesPullingTime="1.025633621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:50:22.816190871 +0000 UTC m=+637.606278804" watchObservedRunningTime="2026-07-13 16:50:30.662450873 +0000 UTC m=+645.452538806" Jul 13 16:50:30.822950 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:30.822919 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ht6w5" Jul 13 16:50:31.101559 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.101526 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d"] Jul 13 16:50:31.105223 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.105193 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.108070 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.108030 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 16:50:31.119221 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.119195 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d"] Jul 13 16:50:31.218451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218407 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.218451 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218454 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c95ad49-b766-405d-97c1-cb271df013dd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.218711 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218517 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzjkj\" (UniqueName: \"kubernetes.io/projected/1c95ad49-b766-405d-97c1-cb271df013dd-kube-api-access-kzjkj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.218711 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218557 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.218711 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218610 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.218863 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.218734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319705 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319705 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319704 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c95ad49-b766-405d-97c1-cb271df013dd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzjkj\" (UniqueName: \"kubernetes.io/projected/1c95ad49-b766-405d-97c1-cb271df013dd-kube-api-access-kzjkj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319748 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319766 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.319930 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.319814 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.320233 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.320205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.320233 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.320220 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.320343 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.320308 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.322112 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.322092 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c95ad49-b766-405d-97c1-cb271df013dd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.322289 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.322272 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c95ad49-b766-405d-97c1-cb271df013dd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.329614 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.329589 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzjkj\" (UniqueName: \"kubernetes.io/projected/1c95ad49-b766-405d-97c1-cb271df013dd-kube-api-access-kzjkj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d\" (UID: \"1c95ad49-b766-405d-97c1-cb271df013dd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.416064 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.416021 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:31.550663 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.550637 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d"] Jul 13 16:50:31.552687 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:50:31.552655 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c95ad49_b766_405d_97c1_cb271df013dd.slice/crio-6b03874ac156c743c1db0a231c3832f636258cb551b2185968f7d5ce0bfbaac6 WatchSource:0}: Error finding container 6b03874ac156c743c1db0a231c3832f636258cb551b2185968f7d5ce0bfbaac6: Status 404 returned error can't find the container with id 6b03874ac156c743c1db0a231c3832f636258cb551b2185968f7d5ce0bfbaac6 Jul 13 16:50:31.818684 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.818653 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" event={"ID":"1c95ad49-b766-405d-97c1-cb271df013dd","Type":"ContainerStarted","Data":"d368da48fa1cf47355c4bd3b180653762b8c683fd657a3a7ee254176304b708d"} Jul 13 16:50:31.819004 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.818708 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" event={"ID":"1c95ad49-b766-405d-97c1-cb271df013dd","Type":"ContainerStarted","Data":"6b03874ac156c743c1db0a231c3832f636258cb551b2185968f7d5ce0bfbaac6"} Jul 13 16:50:31.841365 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:31.841315 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" podStartSLOduration=0.648399463 podStartE2EDuration="841.300872ms" totalImagesPullingTime="192.901409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:50:31.838987006 +0000 UTC m=+646.629074942" watchObservedRunningTime="2026-07-13 16:50:31.841300872 +0000 UTC m=+646.631388805" Jul 13 16:50:41.417000 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:41.416954 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:41.417417 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:41.417028 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:41.429618 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:41.429593 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:50:41.879633 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:50:41.879560 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d" Jul 13 16:53:42.987370 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:42.987328 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:53:42.987814 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:42.987552 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-69cbf74464-6q4rt" podUID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" containerName="manager" containerID="cri-o://7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176" gracePeriod=30 Jul 13 16:53:43.237190 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.237166 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:53:43.312431 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.312350 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqb42\" (UniqueName: \"kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42\") pod \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " Jul 13 16:53:43.312594 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.312474 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert\") pod \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\" (UID: \"3fc6b2b5-c8d2-42de-a39f-f87932ee644a\") " Jul 13 16:53:43.314277 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.314251 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42" (OuterVolumeSpecName: "kube-api-access-lqb42") pod "3fc6b2b5-c8d2-42de-a39f-f87932ee644a" (UID: "3fc6b2b5-c8d2-42de-a39f-f87932ee644a"). InnerVolumeSpecName "kube-api-access-lqb42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 16:53:43.314414 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.314390 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3fc6b2b5-c8d2-42de-a39f-f87932ee644a" (UID: "3fc6b2b5-c8d2-42de-a39f-f87932ee644a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 16:53:43.413952 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.413918 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-webhook-cert\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:53:43.413952 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.413946 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqb42\" (UniqueName: \"kubernetes.io/projected/3fc6b2b5-c8d2-42de-a39f-f87932ee644a-kube-api-access-lqb42\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 16:53:43.801985 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.801952 2569 generic.go:356] "Generic (PLEG): container finished" podID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" containerID="7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176" exitCode=0 Jul 13 16:53:43.802185 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.801999 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-6q4rt" Jul 13 16:53:43.802185 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.802013 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-6q4rt" event={"ID":"3fc6b2b5-c8d2-42de-a39f-f87932ee644a","Type":"ContainerDied","Data":"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176"} Jul 13 16:53:43.802185 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.802086 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-6q4rt" event={"ID":"3fc6b2b5-c8d2-42de-a39f-f87932ee644a","Type":"ContainerDied","Data":"a83a0c7e5e6b71313f5e78d7efea25b157180391fecffb0c06efcfa0774fa68f"} Jul 13 16:53:43.802185 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.802108 2569 scope.go:117] "RemoveContainer" containerID="7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176" Jul 13 16:53:43.811218 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.811193 2569 scope.go:117] "RemoveContainer" containerID="7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176" Jul 13 16:53:43.811514 ip-10-0-141-178 kubenswrapper[2569]: E0713 16:53:43.811491 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176\": container with ID starting with 7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176 not found: ID does not exist" containerID="7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176" Jul 13 16:53:43.811564 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.811523 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176"} err="failed to get container status \"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176\": rpc error: code = NotFound desc = could not find container \"7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176\": container with ID starting with 7263500a5ed5d0f45a518c3521a41c1453e023745db0f36e109d19465fab3176 not found: ID does not exist" Jul 13 16:53:43.821453 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.821422 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:53:43.825892 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:43.825867 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-69cbf74464-6q4rt"] Jul 13 16:53:44.612724 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.612674 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-69cbf74464-rtf77"] Jul 13 16:53:44.613236 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.613086 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" containerName="manager" Jul 13 16:53:44.613236 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.613100 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" containerName="manager" Jul 13 16:53:44.613236 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.613160 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" containerName="manager" Jul 13 16:53:44.617447 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.617432 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.621050 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.621011 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 16:53:44.621050 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.621011 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 16:53:44.621238 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.621071 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tqpmc\"" Jul 13 16:53:44.625931 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.625910 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69cbf74464-rtf77"] Jul 13 16:53:44.726239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.726194 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4911f1f-b410-400d-b93f-5a83774587c8-webhook-cert\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.726239 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.726250 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dscll\" (UniqueName: \"kubernetes.io/projected/a4911f1f-b410-400d-b93f-5a83774587c8-kube-api-access-dscll\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.827608 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.827569 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4911f1f-b410-400d-b93f-5a83774587c8-webhook-cert\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.827792 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.827620 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dscll\" (UniqueName: \"kubernetes.io/projected/a4911f1f-b410-400d-b93f-5a83774587c8-kube-api-access-dscll\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.829926 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.829907 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4911f1f-b410-400d-b93f-5a83774587c8-webhook-cert\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.836394 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.836370 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dscll\" (UniqueName: \"kubernetes.io/projected/a4911f1f-b410-400d-b93f-5a83774587c8-kube-api-access-dscll\") pod \"maas-controller-69cbf74464-rtf77\" (UID: \"a4911f1f-b410-400d-b93f-5a83774587c8\") " pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:44.929670 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:44.929638 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:45.061858 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.061822 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-69cbf74464-rtf77"] Jul 13 16:53:45.065186 ip-10-0-141-178 kubenswrapper[2569]: W0713 16:53:45.065162 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4911f1f_b410_400d_b93f_5a83774587c8.slice/crio-6748a823991e6a5e74fa9f7542f22a894c243a36d822e90fad14dc3041da5abf WatchSource:0}: Error finding container 6748a823991e6a5e74fa9f7542f22a894c243a36d822e90fad14dc3041da5abf: Status 404 returned error can't find the container with id 6748a823991e6a5e74fa9f7542f22a894c243a36d822e90fad14dc3041da5abf Jul 13 16:53:45.725948 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.725914 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc6b2b5-c8d2-42de-a39f-f87932ee644a" path="/var/lib/kubelet/pods/3fc6b2b5-c8d2-42de-a39f-f87932ee644a/volumes" Jul 13 16:53:45.813864 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.813782 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-rtf77" event={"ID":"a4911f1f-b410-400d-b93f-5a83774587c8","Type":"ContainerStarted","Data":"40e0729de012b995709847b1ab822abc866299200faa735009187e5838bbdbe7"} Jul 13 16:53:45.813864 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.813836 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-69cbf74464-rtf77" event={"ID":"a4911f1f-b410-400d-b93f-5a83774587c8","Type":"ContainerStarted","Data":"6748a823991e6a5e74fa9f7542f22a894c243a36d822e90fad14dc3041da5abf"} Jul 13 16:53:45.814084 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.813908 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:53:45.831005 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:45.830953 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-69cbf74464-rtf77" podStartSLOduration=1.38665273 podStartE2EDuration="1.830936472s" totalImagesPullingTime="444.283742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:53:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:53:45.830718187 +0000 UTC m=+840.620806135" watchObservedRunningTime="2026-07-13 16:53:45.830936472 +0000 UTC m=+840.621024407" Jul 13 16:53:56.821524 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:53:56.821490 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-69cbf74464-rtf77" Jul 13 16:54:45.727295 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:54:45.727215 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:54:45.730981 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:54:45.730960 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:54:45.731375 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:54:45.731359 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:54:45.734534 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:54:45.734516 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:59:45.762689 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:59:45.762660 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:59:45.763233 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:59:45.762768 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 16:59:45.766022 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:59:45.766003 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 16:59:45.766172 ip-10-0-141-178 kubenswrapper[2569]: I0713 16:59:45.766093 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:04:45.793980 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:04:45.793952 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:04:45.795442 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:04:45.795423 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:04:45.797592 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:04:45.797571 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:04:45.798794 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:04:45.798778 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:05:58.232452 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.232412 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:05:58.236511 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.236489 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.244991 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.244972 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a5ed8a38-serving-cert\"" Jul 13 17:05:58.259488 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.259460 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:05:58.324464 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.324438 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.324604 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.324478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ljr\" (UniqueName: \"kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.324604 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.324504 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.425131 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.425100 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.425293 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.425182 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.425293 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.425211 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ljr\" (UniqueName: \"kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.425293 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:05:58.425251 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-serving-cert: secret "maas-api-e2e-ait-a5ed8a38-serving-cert" not found Jul 13 17:05:58.425425 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:05:58.425327 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls podName:8417922f-82c5-42a0-af46-6be9086612a4 nodeName:}" failed. No retries permitted until 2026-07-13 17:05:58.925307401 +0000 UTC m=+1573.715395315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls") pod "maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" (UID: "8417922f-82c5-42a0-af46-6be9086612a4") : secret "maas-api-e2e-ait-a5ed8a38-serving-cert" not found Jul 13 17:05:58.425805 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.425789 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.447812 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.447778 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ljr\" (UniqueName: \"kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.930091 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.930051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:58.932465 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:58.932438 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") pod \"maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:59.147721 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:59.147687 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:05:59.301578 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:59.301547 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:05:59.302989 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:05:59.302962 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8417922f_82c5_42a0_af46_6be9086612a4.slice/crio-c67796b671f5dc9cfedc5cb120299d8a20195cd67731a08d5bf5f406e39a113e WatchSource:0}: Error finding container c67796b671f5dc9cfedc5cb120299d8a20195cd67731a08d5bf5f406e39a113e: Status 404 returned error can't find the container with id c67796b671f5dc9cfedc5cb120299d8a20195cd67731a08d5bf5f406e39a113e Jul 13 17:05:59.304384 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:59.304366 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:05:59.551087 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:05:59.550986 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" event={"ID":"8417922f-82c5-42a0-af46-6be9086612a4","Type":"ContainerStarted","Data":"c67796b671f5dc9cfedc5cb120299d8a20195cd67731a08d5bf5f406e39a113e"} Jul 13 17:06:00.557485 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:00.557409 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" event={"ID":"8417922f-82c5-42a0-af46-6be9086612a4","Type":"ContainerStarted","Data":"251f65f1aeedf6eaf252036fd6e9fe2ec516023d7ba08bc211ef26290d5168d1"} Jul 13 17:06:00.557887 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:00.557546 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:06:00.585906 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:00.585797 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" podStartSLOduration=1.571971123 podStartE2EDuration="2.585780706s" totalImagesPullingTime="1.013809583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:00.584977604 +0000 UTC m=+1575.375065539" watchObservedRunningTime="2026-07-13 17:06:00.585780706 +0000 UTC m=+1575.375868640" Jul 13 17:06:06.568562 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:06.568528 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:06:13.400311 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.400272 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:06:13.400807 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.400564 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" podUID="8417922f-82c5-42a0-af46-6be9086612a4" containerName="maas-api" containerID="cri-o://251f65f1aeedf6eaf252036fd6e9fe2ec516023d7ba08bc211ef26290d5168d1" gracePeriod=30 Jul 13 17:06:13.632803 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.632772 2569 generic.go:356] "Generic (PLEG): container finished" podID="8417922f-82c5-42a0-af46-6be9086612a4" containerID="251f65f1aeedf6eaf252036fd6e9fe2ec516023d7ba08bc211ef26290d5168d1" exitCode=0 Jul 13 17:06:13.632916 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.632841 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" event={"ID":"8417922f-82c5-42a0-af46-6be9086612a4","Type":"ContainerDied","Data":"251f65f1aeedf6eaf252036fd6e9fe2ec516023d7ba08bc211ef26290d5168d1"} Jul 13 17:06:13.651316 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.651264 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:06:13.759218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.759187 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca\") pod \"8417922f-82c5-42a0-af46-6be9086612a4\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " Jul 13 17:06:13.759218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.759219 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7ljr\" (UniqueName: \"kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr\") pod \"8417922f-82c5-42a0-af46-6be9086612a4\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " Jul 13 17:06:13.759466 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.759271 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") pod \"8417922f-82c5-42a0-af46-6be9086612a4\" (UID: \"8417922f-82c5-42a0-af46-6be9086612a4\") " Jul 13 17:06:13.759604 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.759578 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8417922f-82c5-42a0-af46-6be9086612a4" (UID: "8417922f-82c5-42a0-af46-6be9086612a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:06:13.759806 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.759788 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8417922f-82c5-42a0-af46-6be9086612a4-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:06:13.761466 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.761436 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8417922f-82c5-42a0-af46-6be9086612a4" (UID: "8417922f-82c5-42a0-af46-6be9086612a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:06:13.761546 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.761470 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr" (OuterVolumeSpecName: "kube-api-access-p7ljr") pod "8417922f-82c5-42a0-af46-6be9086612a4" (UID: "8417922f-82c5-42a0-af46-6be9086612a4"). InnerVolumeSpecName "kube-api-access-p7ljr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:06:13.860703 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.860671 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7ljr\" (UniqueName: \"kubernetes.io/projected/8417922f-82c5-42a0-af46-6be9086612a4-kube-api-access-p7ljr\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:06:13.860703 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:13.860700 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8417922f-82c5-42a0-af46-6be9086612a4-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:06:14.639314 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:14.639276 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" Jul 13 17:06:14.639314 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:14.639301 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm" event={"ID":"8417922f-82c5-42a0-af46-6be9086612a4","Type":"ContainerDied","Data":"c67796b671f5dc9cfedc5cb120299d8a20195cd67731a08d5bf5f406e39a113e"} Jul 13 17:06:14.639815 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:14.639359 2569 scope.go:117] "RemoveContainer" containerID="251f65f1aeedf6eaf252036fd6e9fe2ec516023d7ba08bc211ef26290d5168d1" Jul 13 17:06:14.663166 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:14.663139 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:06:14.669681 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:14.669662 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a5ed8a38-687445cb89-pq8cm"] Jul 13 17:06:15.720211 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:15.720174 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8417922f-82c5-42a0-af46-6be9086612a4" path="/var/lib/kubelet/pods/8417922f-82c5-42a0-af46-6be9086612a4/volumes" Jul 13 17:06:30.468951 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.468914 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:06:30.469436 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.469297 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8417922f-82c5-42a0-af46-6be9086612a4" containerName="maas-api" Jul 13 17:06:30.469436 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.469310 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8417922f-82c5-42a0-af46-6be9086612a4" containerName="maas-api" Jul 13 17:06:30.469436 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.469359 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8417922f-82c5-42a0-af46-6be9086612a4" containerName="maas-api" Jul 13 17:06:30.472551 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.472526 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.475360 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.475337 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-51bf4736-serving-cert\"" Jul 13 17:06:30.483682 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.483654 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:06:30.601627 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.601590 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.601836 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.601645 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.601836 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.601764 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-642mv\" (UniqueName: \"kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.703268 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.703231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.703444 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.703294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-642mv\" (UniqueName: \"kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.703444 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.703347 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.704122 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.704096 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.705702 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.705683 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.712698 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.712676 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-642mv\" (UniqueName: \"kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv\") pod \"maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.785617 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.785536 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:30.918904 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:30.918875 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:06:30.920746 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:06:30.920718 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52fa8eb5_bb93_4eeb_86d7_99a5b2bfedda.slice/crio-83655235cb7faedb44efc7653d1cad8239d54931af4eab9350fdc4ee6e1cbb98 WatchSource:0}: Error finding container 83655235cb7faedb44efc7653d1cad8239d54931af4eab9350fdc4ee6e1cbb98: Status 404 returned error can't find the container with id 83655235cb7faedb44efc7653d1cad8239d54931af4eab9350fdc4ee6e1cbb98 Jul 13 17:06:31.726248 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:31.726216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" event={"ID":"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda","Type":"ContainerStarted","Data":"83655235cb7faedb44efc7653d1cad8239d54931af4eab9350fdc4ee6e1cbb98"} Jul 13 17:06:32.732493 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:32.732452 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" event={"ID":"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda","Type":"ContainerStarted","Data":"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f"} Jul 13 17:06:32.732975 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:32.732606 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:06:32.752288 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:32.752247 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" podStartSLOduration=1.752891173 podStartE2EDuration="2.752234347s" totalImagesPullingTime="999.343174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:32.750855967 +0000 UTC m=+1607.540943901" watchObservedRunningTime="2026-07-13 17:06:32.752234347 +0000 UTC m=+1607.542322280" Jul 13 17:06:38.742844 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:06:38.742767 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:07:40.683795 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:40.683761 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:07:40.684457 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:40.683965 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" podUID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" containerName="maas-api" containerID="cri-o://6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f" gracePeriod=30 Jul 13 17:07:40.943301 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:40.943242 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:07:41.084802 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.084768 2569 generic.go:356] "Generic (PLEG): container finished" podID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" containerID="6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f" exitCode=0 Jul 13 17:07:41.084968 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.084816 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" Jul 13 17:07:41.084968 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.084848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" event={"ID":"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda","Type":"ContainerDied","Data":"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f"} Jul 13 17:07:41.084968 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.084899 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w" event={"ID":"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda","Type":"ContainerDied","Data":"83655235cb7faedb44efc7653d1cad8239d54931af4eab9350fdc4ee6e1cbb98"} Jul 13 17:07:41.084968 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.084922 2569 scope.go:117] "RemoveContainer" containerID="6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f" Jul 13 17:07:41.093455 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.093437 2569 scope.go:117] "RemoveContainer" containerID="6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f" Jul 13 17:07:41.093682 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:07:41.093662 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f\": container with ID starting with 6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f not found: ID does not exist" containerID="6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f" Jul 13 17:07:41.093756 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.093695 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f"} err="failed to get container status \"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f\": rpc error: code = NotFound desc = could not find container \"6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f\": container with ID starting with 6b356a53d7bd5e6f8105275083723be40d54a1a0d233fc8862f85326121f662f not found: ID does not exist" Jul 13 17:07:41.099015 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.098988 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls\") pod \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " Jul 13 17:07:41.099122 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.099057 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca\") pod \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " Jul 13 17:07:41.099209 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.099194 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-642mv\" (UniqueName: \"kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv\") pod \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\" (UID: \"52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda\") " Jul 13 17:07:41.099428 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.099402 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" (UID: "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:07:41.100965 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.100946 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" (UID: "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:07:41.101098 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.101080 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv" (OuterVolumeSpecName: "kube-api-access-642mv") pod "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" (UID: "52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda"). InnerVolumeSpecName "kube-api-access-642mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:07:41.200724 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.200658 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-642mv\" (UniqueName: \"kubernetes.io/projected/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-kube-api-access-642mv\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:07:41.200724 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.200682 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:07:41.200724 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.200693 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:07:41.408671 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.408637 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:07:41.412629 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.412588 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-51bf4736-5d597fccc9-prl5w"] Jul 13 17:07:41.719916 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:41.719879 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" path="/var/lib/kubelet/pods/52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda/volumes" Jul 13 17:07:49.082456 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.082424 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:07:49.082868 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.082785 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" containerName="maas-api" Jul 13 17:07:49.082868 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.082796 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" containerName="maas-api" Jul 13 17:07:49.082868 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.082847 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="52fa8eb5-bb93-4eeb-86d7-99a5b2bfedda" containerName="maas-api" Jul 13 17:07:49.086013 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.085988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.088483 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.088460 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-97e65980-serving-cert\"" Jul 13 17:07:49.096807 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.096777 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:07:49.167636 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.167597 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.167636 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.167632 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmb6r\" (UniqueName: \"kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.167873 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.167696 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.268943 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.268899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.269162 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.268985 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.269162 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.269015 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmb6r\" (UniqueName: \"kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.269855 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.269825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.271538 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.271509 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.282612 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.282583 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmb6r\" (UniqueName: \"kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r\") pod \"maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.399846 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.399820 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:49.549480 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:49.549452 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:07:49.551306 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:07:49.551268 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb117ef_a761_4ab9_94bf_9d039c846884.slice/crio-0ea7bafb90787232fa344ec3dd6e0853f979d7aab5864d7cc5040be45bbe2a23 WatchSource:0}: Error finding container 0ea7bafb90787232fa344ec3dd6e0853f979d7aab5864d7cc5040be45bbe2a23: Status 404 returned error can't find the container with id 0ea7bafb90787232fa344ec3dd6e0853f979d7aab5864d7cc5040be45bbe2a23 Jul 13 17:07:50.131561 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:50.131525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" event={"ID":"acb117ef-a761-4ab9-94bf-9d039c846884","Type":"ContainerStarted","Data":"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d"} Jul 13 17:07:50.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:50.131595 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" event={"ID":"acb117ef-a761-4ab9-94bf-9d039c846884","Type":"ContainerStarted","Data":"0ea7bafb90787232fa344ec3dd6e0853f979d7aab5864d7cc5040be45bbe2a23"} Jul 13 17:07:50.131949 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:50.131619 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:07:50.150687 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:50.150639 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" podStartSLOduration=0.77882753 podStartE2EDuration="1.150624366s" totalImagesPullingTime="371.796836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:07:50.148451433 +0000 UTC m=+1684.938539367" watchObservedRunningTime="2026-07-13 17:07:50.150624366 +0000 UTC m=+1684.940712300" Jul 13 17:07:56.142487 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:07:56.142454 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:08:04.803624 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:04.803593 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:08:04.804024 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:04.803820 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" podUID="acb117ef-a761-4ab9-94bf-9d039c846884" containerName="maas-api" containerID="cri-o://75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d" gracePeriod=30 Jul 13 17:08:05.054561 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.054501 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:08:05.103699 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.103668 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmb6r\" (UniqueName: \"kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r\") pod \"acb117ef-a761-4ab9-94bf-9d039c846884\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " Jul 13 17:08:05.103847 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.103736 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls\") pod \"acb117ef-a761-4ab9-94bf-9d039c846884\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " Jul 13 17:08:05.103847 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.103762 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca\") pod \"acb117ef-a761-4ab9-94bf-9d039c846884\" (UID: \"acb117ef-a761-4ab9-94bf-9d039c846884\") " Jul 13 17:08:05.104170 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.104145 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "acb117ef-a761-4ab9-94bf-9d039c846884" (UID: "acb117ef-a761-4ab9-94bf-9d039c846884"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:08:05.105778 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.105754 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "acb117ef-a761-4ab9-94bf-9d039c846884" (UID: "acb117ef-a761-4ab9-94bf-9d039c846884"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:08:05.105846 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.105794 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r" (OuterVolumeSpecName: "kube-api-access-qmb6r") pod "acb117ef-a761-4ab9-94bf-9d039c846884" (UID: "acb117ef-a761-4ab9-94bf-9d039c846884"). InnerVolumeSpecName "kube-api-access-qmb6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:08:05.205083 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.205029 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmb6r\" (UniqueName: \"kubernetes.io/projected/acb117ef-a761-4ab9-94bf-9d039c846884-kube-api-access-qmb6r\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:08:05.205083 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.205078 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb117ef-a761-4ab9-94bf-9d039c846884-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:08:05.205083 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.205088 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb117ef-a761-4ab9-94bf-9d039c846884-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:08:05.215955 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.215924 2569 generic.go:356] "Generic (PLEG): container finished" podID="acb117ef-a761-4ab9-94bf-9d039c846884" containerID="75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d" exitCode=0 Jul 13 17:08:05.216109 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.215967 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" Jul 13 17:08:05.216109 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.216008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" event={"ID":"acb117ef-a761-4ab9-94bf-9d039c846884","Type":"ContainerDied","Data":"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d"} Jul 13 17:08:05.216109 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.216081 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd" event={"ID":"acb117ef-a761-4ab9-94bf-9d039c846884","Type":"ContainerDied","Data":"0ea7bafb90787232fa344ec3dd6e0853f979d7aab5864d7cc5040be45bbe2a23"} Jul 13 17:08:05.216109 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.216108 2569 scope.go:117] "RemoveContainer" containerID="75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d" Jul 13 17:08:05.226280 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.226261 2569 scope.go:117] "RemoveContainer" containerID="75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d" Jul 13 17:08:05.226565 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:08:05.226527 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d\": container with ID starting with 75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d not found: ID does not exist" containerID="75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d" Jul 13 17:08:05.226666 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.226566 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d"} err="failed to get container status \"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d\": rpc error: code = NotFound desc = could not find container \"75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d\": container with ID starting with 75a5a5c900f783c015d93e8c34681ad371650fa592a8ee856ad2a8d324f9fe7d not found: ID does not exist" Jul 13 17:08:05.238840 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.238815 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:08:05.245000 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.244973 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-97e65980-66d5dbcbcf-mmbcd"] Jul 13 17:08:05.726210 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:05.726170 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb117ef-a761-4ab9-94bf-9d039c846884" path="/var/lib/kubelet/pods/acb117ef-a761-4ab9-94bf-9d039c846884/volumes" Jul 13 17:08:21.965825 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.965788 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:08:21.966377 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.966360 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb117ef-a761-4ab9-94bf-9d039c846884" containerName="maas-api" Jul 13 17:08:21.966501 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.966380 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb117ef-a761-4ab9-94bf-9d039c846884" containerName="maas-api" Jul 13 17:08:21.966501 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.966487 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb117ef-a761-4ab9-94bf-9d039c846884" containerName="maas-api" Jul 13 17:08:21.970005 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.969983 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:21.972512 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.972488 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1890c174-serving-cert\"" Jul 13 17:08:21.980581 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:21.980557 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:08:22.044437 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.044396 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.044603 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.044455 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.044603 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.044498 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbwh\" (UniqueName: \"kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.145760 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.145730 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.145930 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.145781 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbwh\" (UniqueName: \"kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.145930 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.145826 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.145930 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:08:22.145889 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-serving-cert: secret "maas-api-e2e-derive-1890c174-serving-cert" not found Jul 13 17:08:22.146047 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:08:22.145993 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls podName:6dc34c56-2f6a-441c-8ba9-1861f309ef59 nodeName:}" failed. No retries permitted until 2026-07-13 17:08:22.645975886 +0000 UTC m=+1717.436063803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls") pod "maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" (UID: "6dc34c56-2f6a-441c-8ba9-1861f309ef59") : secret "maas-api-e2e-derive-1890c174-serving-cert" not found Jul 13 17:08:22.146548 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.146530 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.154820 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.154794 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbwh\" (UniqueName: \"kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.650120 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.650086 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.652552 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.652528 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") pod \"maas-api-e2e-derive-1890c174-7f59845d84-9dp2x\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:22.882934 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:22.882898 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:23.019243 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:23.019220 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:08:23.021177 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:08:23.021142 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc34c56_2f6a_441c_8ba9_1861f309ef59.slice/crio-3a760abee9daa1a8c8fd52fccc01d079aac12c980a43cb0659aa35fe58d28c7f WatchSource:0}: Error finding container 3a760abee9daa1a8c8fd52fccc01d079aac12c980a43cb0659aa35fe58d28c7f: Status 404 returned error can't find the container with id 3a760abee9daa1a8c8fd52fccc01d079aac12c980a43cb0659aa35fe58d28c7f Jul 13 17:08:23.309652 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:23.309570 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" event={"ID":"6dc34c56-2f6a-441c-8ba9-1861f309ef59","Type":"ContainerStarted","Data":"3a760abee9daa1a8c8fd52fccc01d079aac12c980a43cb0659aa35fe58d28c7f"} Jul 13 17:08:24.317368 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:24.317333 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" event={"ID":"6dc34c56-2f6a-441c-8ba9-1861f309ef59","Type":"ContainerStarted","Data":"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620"} Jul 13 17:08:24.317759 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:24.317465 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:08:24.335185 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:24.335144 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" podStartSLOduration=2.645365102 podStartE2EDuration="3.335131602s" totalImagesPullingTime="689.7665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:08:24.334297991 +0000 UTC m=+1719.124385926" watchObservedRunningTime="2026-07-13 17:08:24.335131602 +0000 UTC m=+1719.125219536" Jul 13 17:08:30.328991 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:08:30.328960 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:09:26.864798 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:26.864762 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:09:26.865341 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:26.865018 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" podUID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" containerName="maas-api" containerID="cri-o://372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620" gracePeriod=30 Jul 13 17:09:27.115725 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.115666 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:09:27.210588 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.210542 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctbwh\" (UniqueName: \"kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh\") pod \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " Jul 13 17:09:27.210777 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.210673 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca\") pod \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " Jul 13 17:09:27.210777 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.210708 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") pod \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\" (UID: \"6dc34c56-2f6a-441c-8ba9-1861f309ef59\") " Jul 13 17:09:27.211081 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.211021 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6dc34c56-2f6a-441c-8ba9-1861f309ef59" (UID: "6dc34c56-2f6a-441c-8ba9-1861f309ef59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:09:27.212688 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.212666 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6dc34c56-2f6a-441c-8ba9-1861f309ef59" (UID: "6dc34c56-2f6a-441c-8ba9-1861f309ef59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:09:27.212798 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.212775 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh" (OuterVolumeSpecName: "kube-api-access-ctbwh") pod "6dc34c56-2f6a-441c-8ba9-1861f309ef59" (UID: "6dc34c56-2f6a-441c-8ba9-1861f309ef59"). InnerVolumeSpecName "kube-api-access-ctbwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:09:27.312303 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.312271 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctbwh\" (UniqueName: \"kubernetes.io/projected/6dc34c56-2f6a-441c-8ba9-1861f309ef59-kube-api-access-ctbwh\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:09:27.312303 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.312298 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dc34c56-2f6a-441c-8ba9-1861f309ef59-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:09:27.312514 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.312312 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6dc34c56-2f6a-441c-8ba9-1861f309ef59-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:09:27.641027 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.640992 2569 generic.go:356] "Generic (PLEG): container finished" podID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" containerID="372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620" exitCode=0 Jul 13 17:09:27.641218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.641033 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" Jul 13 17:09:27.641218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.641078 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" event={"ID":"6dc34c56-2f6a-441c-8ba9-1861f309ef59","Type":"ContainerDied","Data":"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620"} Jul 13 17:09:27.641218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.641134 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x" event={"ID":"6dc34c56-2f6a-441c-8ba9-1861f309ef59","Type":"ContainerDied","Data":"3a760abee9daa1a8c8fd52fccc01d079aac12c980a43cb0659aa35fe58d28c7f"} Jul 13 17:09:27.641218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.641156 2569 scope.go:117] "RemoveContainer" containerID="372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620" Jul 13 17:09:27.652718 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.652695 2569 scope.go:117] "RemoveContainer" containerID="372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620" Jul 13 17:09:27.653115 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:09:27.653093 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620\": container with ID starting with 372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620 not found: ID does not exist" containerID="372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620" Jul 13 17:09:27.653229 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.653122 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620"} err="failed to get container status \"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620\": rpc error: code = NotFound desc = could not find container \"372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620\": container with ID starting with 372f7db8ed9b3b126c6eb45f64dd5314b54a848b6561d9f301b331374de84620 not found: ID does not exist" Jul 13 17:09:27.663119 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.663097 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:09:27.668068 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.668028 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1890c174-7f59845d84-9dp2x"] Jul 13 17:09:27.720241 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:27.720216 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" path="/var/lib/kubelet/pods/6dc34c56-2f6a-441c-8ba9-1861f309ef59/volumes" Jul 13 17:09:45.834723 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:45.834697 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:09:45.838426 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:45.838401 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:09:45.838806 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:45.838781 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:09:45.842258 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:09:45.842239 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:11:22.504801 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.504764 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:22.507157 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.505316 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" containerName="maas-api" Jul 13 17:11:22.507157 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.505337 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" containerName="maas-api" Jul 13 17:11:22.507157 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.505455 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dc34c56-2f6a-441c-8ba9-1861f309ef59" containerName="maas-api" Jul 13 17:11:22.508504 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.508483 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.511142 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.511119 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-45a43c1d-serving-cert\"" Jul 13 17:11:22.518239 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.518214 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:22.678026 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.677987 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkd4g\" (UniqueName: \"kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.678026 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.678059 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.678320 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.678128 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.779286 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.779203 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkd4g\" (UniqueName: \"kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.779286 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.779258 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.779467 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.779301 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.779467 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:11:22.779418 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-serving-cert: secret "maas-api-e2e-mt-45a43c1d-serving-cert" not found Jul 13 17:11:22.779541 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:11:22.779483 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls podName:bccaccf7-9ebe-42e8-8511-a6949ad7f3ba nodeName:}" failed. No retries permitted until 2026-07-13 17:11:23.279468026 +0000 UTC m=+1898.069555938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls") pod "maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" (UID: "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba") : secret "maas-api-e2e-mt-45a43c1d-serving-cert" not found Jul 13 17:11:22.779967 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.779949 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:22.792533 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:22.792501 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkd4g\" (UniqueName: \"kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:23.284196 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:23.284165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:23.286835 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:23.286810 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") pod \"maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:23.420528 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:23.420488 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:23.562976 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:23.562948 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:23.564808 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:11:23.564782 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccaccf7_9ebe_42e8_8511_a6949ad7f3ba.slice/crio-ba4ad7d3a4c13591051de49ef66016ee872e0f62c930a55f9547f4f75009772f WatchSource:0}: Error finding container ba4ad7d3a4c13591051de49ef66016ee872e0f62c930a55f9547f4f75009772f: Status 404 returned error can't find the container with id ba4ad7d3a4c13591051de49ef66016ee872e0f62c930a55f9547f4f75009772f Jul 13 17:11:23.566233 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:23.566216 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:11:24.242119 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:24.242087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" event={"ID":"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba","Type":"ContainerStarted","Data":"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d"} Jul 13 17:11:24.242299 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:24.242137 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" event={"ID":"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba","Type":"ContainerStarted","Data":"ba4ad7d3a4c13591051de49ef66016ee872e0f62c930a55f9547f4f75009772f"} Jul 13 17:11:24.242299 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:24.242217 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:24.260802 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:24.260753 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" podStartSLOduration=1.7228599230000001 podStartE2EDuration="2.260736136s" totalImagesPullingTime="537.876213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:11:24.259999517 +0000 UTC m=+1899.050087451" watchObservedRunningTime="2026-07-13 17:11:24.260736136 +0000 UTC m=+1899.050824069" Jul 13 17:11:30.252293 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:30.252263 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:44.932002 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:44.931968 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:44.932559 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:44.932238 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" podUID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" containerName="maas-api" containerID="cri-o://e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d" gracePeriod=30 Jul 13 17:11:45.183049 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.182971 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:45.270906 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.270872 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkd4g\" (UniqueName: \"kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g\") pod \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " Jul 13 17:11:45.270906 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.270905 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca\") pod \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " Jul 13 17:11:45.271133 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.270943 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") pod \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\" (UID: \"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba\") " Jul 13 17:11:45.271333 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.271294 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" (UID: "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:11:45.273031 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.273010 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g" (OuterVolumeSpecName: "kube-api-access-pkd4g") pod "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" (UID: "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba"). InnerVolumeSpecName "kube-api-access-pkd4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:11:45.273117 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.273022 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" (UID: "bccaccf7-9ebe-42e8-8511-a6949ad7f3ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:11:45.347966 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.347935 2569 generic.go:356] "Generic (PLEG): container finished" podID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" containerID="e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d" exitCode=0 Jul 13 17:11:45.348153 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.347986 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" Jul 13 17:11:45.348153 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.348020 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" event={"ID":"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba","Type":"ContainerDied","Data":"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d"} Jul 13 17:11:45.348153 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.348093 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw" event={"ID":"bccaccf7-9ebe-42e8-8511-a6949ad7f3ba","Type":"ContainerDied","Data":"ba4ad7d3a4c13591051de49ef66016ee872e0f62c930a55f9547f4f75009772f"} Jul 13 17:11:45.348153 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.348117 2569 scope.go:117] "RemoveContainer" containerID="e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d" Jul 13 17:11:45.360079 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.360057 2569 scope.go:117] "RemoveContainer" containerID="e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d" Jul 13 17:11:45.360363 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:11:45.360344 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d\": container with ID starting with e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d not found: ID does not exist" containerID="e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d" Jul 13 17:11:45.360416 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.360372 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d"} err="failed to get container status \"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d\": rpc error: code = NotFound desc = could not find container \"e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d\": container with ID starting with e45b4387cf32f8413ebea64fa14bcca28633165c9b2629f9b1ec282f59312d2d not found: ID does not exist" Jul 13 17:11:45.371851 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.371794 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkd4g\" (UniqueName: \"kubernetes.io/projected/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-kube-api-access-pkd4g\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:11:45.371851 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.371820 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:11:45.371851 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.371830 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:11:45.374458 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.374435 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:45.380389 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.380370 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-45a43c1d-5d8755589b-6rpqw"] Jul 13 17:11:45.721096 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:11:45.721067 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" path="/var/lib/kubelet/pods/bccaccf7-9ebe-42e8-8511-a6949ad7f3ba/volumes" Jul 13 17:12:25.813815 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.813780 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:12:25.814300 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.814173 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" containerName="maas-api" Jul 13 17:12:25.814300 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.814185 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" containerName="maas-api" Jul 13 17:12:25.814300 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.814262 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccaccf7-9ebe-42e8-8511-a6949ad7f3ba" containerName="maas-api" Jul 13 17:12:25.817314 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.817295 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.821759 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.821734 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-54358301-serving-cert\"" Jul 13 17:12:25.826404 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.826374 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:12:25.828231 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.828210 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.828345 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.828245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrx4\" (UniqueName: \"kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.828418 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.828378 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.929786 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.929743 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.929971 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.929826 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.929971 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.929851 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrx4\" (UniqueName: \"kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.929971 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:12:25.929962 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-54358301-serving-cert: secret "maas-api-e2e-mt-54358301-serving-cert" not found Jul 13 17:12:25.930197 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:12:25.930053 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls podName:79914754-a8d3-483f-b5a0-8c1b8877fa4d nodeName:}" failed. No retries permitted until 2026-07-13 17:12:26.430014722 +0000 UTC m=+1961.220102644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls") pod "maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" (UID: "79914754-a8d3-483f-b5a0-8c1b8877fa4d") : secret "maas-api-e2e-mt-54358301-serving-cert" not found Jul 13 17:12:25.930440 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.930421 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:25.940622 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:25.940587 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrx4\" (UniqueName: \"kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:26.434495 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:26.434446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:26.436896 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:26.436862 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") pod \"maas-api-e2e-mt-54358301-66d8677d8b-jxlrj\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:26.729741 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:26.729652 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:26.869165 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:26.869138 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:12:26.870760 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:12:26.870737 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79914754_a8d3_483f_b5a0_8c1b8877fa4d.slice/crio-11ba03319fcfc54215cf76d59b95b4449eaa10849c55ccb821b0a74a652f99b8 WatchSource:0}: Error finding container 11ba03319fcfc54215cf76d59b95b4449eaa10849c55ccb821b0a74a652f99b8: Status 404 returned error can't find the container with id 11ba03319fcfc54215cf76d59b95b4449eaa10849c55ccb821b0a74a652f99b8 Jul 13 17:12:27.571434 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:27.571397 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" event={"ID":"79914754-a8d3-483f-b5a0-8c1b8877fa4d","Type":"ContainerStarted","Data":"11ba03319fcfc54215cf76d59b95b4449eaa10849c55ccb821b0a74a652f99b8"} Jul 13 17:12:28.577979 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:28.577941 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" event={"ID":"79914754-a8d3-483f-b5a0-8c1b8877fa4d","Type":"ContainerStarted","Data":"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3"} Jul 13 17:12:28.578435 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:28.578009 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:12:28.594560 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:28.594502 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" podStartSLOduration=2.614783144 podStartE2EDuration="3.59448146s" totalImagesPullingTime="979.698316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:12:28.594249742 +0000 UTC m=+1963.384337677" watchObservedRunningTime="2026-07-13 17:12:28.59448146 +0000 UTC m=+1963.384569376" Jul 13 17:12:34.588126 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:12:34.588077 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:13:29.562545 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.562507 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:13:29.563007 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.562782 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerName="maas-api" containerID="cri-o://14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3" gracePeriod=30 Jul 13 17:13:29.583218 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.583168 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.52:8443/health\": dial tcp 10.132.0.52:8443: connect: connection refused" Jul 13 17:13:29.813855 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.813798 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:13:29.852858 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.852832 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") pod \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " Jul 13 17:13:29.853032 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.852870 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca\") pod \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " Jul 13 17:13:29.853032 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.852910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfrx4\" (UniqueName: \"kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4\") pod \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\" (UID: \"79914754-a8d3-483f-b5a0-8c1b8877fa4d\") " Jul 13 17:13:29.853348 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.853318 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79914754-a8d3-483f-b5a0-8c1b8877fa4d" (UID: "79914754-a8d3-483f-b5a0-8c1b8877fa4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:13:29.854946 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.854917 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4" (OuterVolumeSpecName: "kube-api-access-sfrx4") pod "79914754-a8d3-483f-b5a0-8c1b8877fa4d" (UID: "79914754-a8d3-483f-b5a0-8c1b8877fa4d"). InnerVolumeSpecName "kube-api-access-sfrx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:13:29.855055 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.854947 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79914754-a8d3-483f-b5a0-8c1b8877fa4d" (UID: "79914754-a8d3-483f-b5a0-8c1b8877fa4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:13:29.892509 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.892476 2569 generic.go:356] "Generic (PLEG): container finished" podID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerID="14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3" exitCode=0 Jul 13 17:13:29.892622 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.892523 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" Jul 13 17:13:29.892622 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.892543 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" event={"ID":"79914754-a8d3-483f-b5a0-8c1b8877fa4d","Type":"ContainerDied","Data":"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3"} Jul 13 17:13:29.892622 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.892595 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj" event={"ID":"79914754-a8d3-483f-b5a0-8c1b8877fa4d","Type":"ContainerDied","Data":"11ba03319fcfc54215cf76d59b95b4449eaa10849c55ccb821b0a74a652f99b8"} Jul 13 17:13:29.892733 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.892624 2569 scope.go:117] "RemoveContainer" containerID="14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3" Jul 13 17:13:29.901932 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.901910 2569 scope.go:117] "RemoveContainer" containerID="14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3" Jul 13 17:13:29.902201 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:13:29.902182 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3\": container with ID starting with 14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3 not found: ID does not exist" containerID="14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3" Jul 13 17:13:29.902256 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.902209 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3"} err="failed to get container status \"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3\": rpc error: code = NotFound desc = could not find container \"14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3\": container with ID starting with 14e7c0b3a4fc64cfa1c15009858c20c956497564e2183094aad0593d616ad9b3 not found: ID does not exist" Jul 13 17:13:29.916401 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.916378 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:13:29.921292 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.921273 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-54358301-66d8677d8b-jxlrj"] Jul 13 17:13:29.953730 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.953711 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79914754-a8d3-483f-b5a0-8c1b8877fa4d-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:13:29.953730 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.953732 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79914754-a8d3-483f-b5a0-8c1b8877fa4d-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:13:29.953849 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:29.953741 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfrx4\" (UniqueName: \"kubernetes.io/projected/79914754-a8d3-483f-b5a0-8c1b8877fa4d-kube-api-access-sfrx4\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:13:31.720346 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:13:31.720313 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" path="/var/lib/kubelet/pods/79914754-a8d3-483f-b5a0-8c1b8877fa4d/volumes" Jul 13 17:14:45.869062 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:14:45.869013 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:14:45.872651 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:14:45.872629 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:14:45.873614 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:14:45.873591 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:14:45.877206 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:14:45.877190 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:15:43.739613 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.739517 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:15:43.740129 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.739864 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerName="maas-api" Jul 13 17:15:43.740129 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.739875 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerName="maas-api" Jul 13 17:15:43.740129 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.739936 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="79914754-a8d3-483f-b5a0-8c1b8877fa4d" containerName="maas-api" Jul 13 17:15:43.743116 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.743098 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.747217 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.747196 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-874fa5-serving-cert\"" Jul 13 17:15:43.755875 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.755854 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:15:43.848014 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.847981 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.848197 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.848028 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrlz\" (UniqueName: \"kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.848197 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.848118 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.949492 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.949442 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrlz\" (UniqueName: \"kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.949492 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.949485 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.949728 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.949569 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.949728 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:15:43.949654 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-serving-cert: secret "maas-api-e2e-inf-a-874fa5-serving-cert" not found Jul 13 17:15:43.949728 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:15:43.949713 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls podName:ba71f282-bf07-4069-8e39-ce3c3c809db4 nodeName:}" failed. No retries permitted until 2026-07-13 17:15:44.449697415 +0000 UTC m=+2159.239785326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls") pod "maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" (UID: "ba71f282-bf07-4069-8e39-ce3c3c809db4") : secret "maas-api-e2e-inf-a-874fa5-serving-cert" not found Jul 13 17:15:43.950202 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.950184 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:43.965203 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:43.965180 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrlz\" (UniqueName: \"kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:44.453154 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:44.453114 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:44.455427 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:44.455399 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") pod \"maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:44.653440 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:44.653410 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:44.795651 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:44.795619 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:15:44.797550 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:15:44.797521 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba71f282_bf07_4069_8e39_ce3c3c809db4.slice/crio-b0b7f160cbeb1e871c7d286f2cf33c55b811c784dc7bd7ec525f46714cd17fcd WatchSource:0}: Error finding container b0b7f160cbeb1e871c7d286f2cf33c55b811c784dc7bd7ec525f46714cd17fcd: Status 404 returned error can't find the container with id b0b7f160cbeb1e871c7d286f2cf33c55b811c784dc7bd7ec525f46714cd17fcd Jul 13 17:15:45.581464 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:45.581429 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" event={"ID":"ba71f282-bf07-4069-8e39-ce3c3c809db4","Type":"ContainerStarted","Data":"b0b7f160cbeb1e871c7d286f2cf33c55b811c784dc7bd7ec525f46714cd17fcd"} Jul 13 17:15:46.587826 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:46.587778 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" event={"ID":"ba71f282-bf07-4069-8e39-ce3c3c809db4","Type":"ContainerStarted","Data":"d52862583460758bc6aad44e5f10fc2b17220b2a3d2412ca0143bbdef512cf10"} Jul 13 17:15:46.588258 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:46.587861 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:46.605711 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:46.605664 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" podStartSLOduration=2.824349408 podStartE2EDuration="3.605638509s" totalImagesPullingTime="781.289101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:15:46.604183694 +0000 UTC m=+2161.394271629" watchObservedRunningTime="2026-07-13 17:15:46.605638509 +0000 UTC m=+2161.395726492" Jul 13 17:15:51.857798 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.857753 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:15:51.860689 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.860662 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:51.863462 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.863434 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0c5523-serving-cert\"" Jul 13 17:15:51.875252 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.875229 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:15:51.933018 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.932988 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:51.933190 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.933084 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hwfw\" (UniqueName: \"kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:51.933190 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:51.933106 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.034386 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.034350 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hwfw\" (UniqueName: \"kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.034386 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.034392 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.034626 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.034483 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.034700 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:15:52.034626 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-serving-cert: secret "maas-api-e2e-inf-b-0c5523-serving-cert" not found Jul 13 17:15:52.034758 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:15:52.034747 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls podName:d7067035-e572-4782-ae72-b658e7ed8e16 nodeName:}" failed. No retries permitted until 2026-07-13 17:15:52.534729017 +0000 UTC m=+2167.324816942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls") pod "maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" (UID: "d7067035-e572-4782-ae72-b658e7ed8e16") : secret "maas-api-e2e-inf-b-0c5523-serving-cert" not found Jul 13 17:15:52.035202 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.035175 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.049811 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.049786 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hwfw\" (UniqueName: \"kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.538997 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.538961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.541424 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.541400 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") pod \"maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.599365 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.599333 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:15:52.771581 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.771544 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:52.908145 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:52.908110 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:15:52.909834 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:15:52.909802 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7067035_e572_4782_ae72_b658e7ed8e16.slice/crio-d028a1d57ace01a1eddd1f05689dcb888adc9e8177758b68c93195d2c089583a WatchSource:0}: Error finding container d028a1d57ace01a1eddd1f05689dcb888adc9e8177758b68c93195d2c089583a: Status 404 returned error can't find the container with id d028a1d57ace01a1eddd1f05689dcb888adc9e8177758b68c93195d2c089583a Jul 13 17:15:53.626471 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:53.626376 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" event={"ID":"d7067035-e572-4782-ae72-b658e7ed8e16","Type":"ContainerStarted","Data":"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538"} Jul 13 17:15:53.626471 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:53.626432 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" event={"ID":"d7067035-e572-4782-ae72-b658e7ed8e16","Type":"ContainerStarted","Data":"d028a1d57ace01a1eddd1f05689dcb888adc9e8177758b68c93195d2c089583a"} Jul 13 17:15:53.626670 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:53.626549 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:15:53.662598 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:53.662509 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" podStartSLOduration=2.222745693 podStartE2EDuration="2.662484065s" totalImagesPullingTime="439.738372ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:15:53.661346016 +0000 UTC m=+2168.451433954" watchObservedRunningTime="2026-07-13 17:15:53.662484065 +0000 UTC m=+2168.452572000" Jul 13 17:15:59.638745 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:15:59.638715 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:17:21.644069 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:21.644019 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:17:21.644678 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:21.644265 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" podUID="ba71f282-bf07-4069-8e39-ce3c3c809db4" containerName="maas-api" containerID="cri-o://d52862583460758bc6aad44e5f10fc2b17220b2a3d2412ca0143bbdef512cf10" gracePeriod=30 Jul 13 17:17:22.090400 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.090268 2569 generic.go:356] "Generic (PLEG): container finished" podID="ba71f282-bf07-4069-8e39-ce3c3c809db4" containerID="d52862583460758bc6aad44e5f10fc2b17220b2a3d2412ca0143bbdef512cf10" exitCode=0 Jul 13 17:17:22.090400 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.090337 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" event={"ID":"ba71f282-bf07-4069-8e39-ce3c3c809db4","Type":"ContainerDied","Data":"d52862583460758bc6aad44e5f10fc2b17220b2a3d2412ca0143bbdef512cf10"} Jul 13 17:17:22.104926 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.104904 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:17:22.196714 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.196628 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvrlz\" (UniqueName: \"kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz\") pod \"ba71f282-bf07-4069-8e39-ce3c3c809db4\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " Jul 13 17:17:22.196714 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.196670 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") pod \"ba71f282-bf07-4069-8e39-ce3c3c809db4\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " Jul 13 17:17:22.196714 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.196687 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca\") pod \"ba71f282-bf07-4069-8e39-ce3c3c809db4\" (UID: \"ba71f282-bf07-4069-8e39-ce3c3c809db4\") " Jul 13 17:17:22.197127 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.197101 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba71f282-bf07-4069-8e39-ce3c3c809db4" (UID: "ba71f282-bf07-4069-8e39-ce3c3c809db4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:17:22.198792 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.198763 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba71f282-bf07-4069-8e39-ce3c3c809db4" (UID: "ba71f282-bf07-4069-8e39-ce3c3c809db4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:17:22.198884 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.198789 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz" (OuterVolumeSpecName: "kube-api-access-xvrlz") pod "ba71f282-bf07-4069-8e39-ce3c3c809db4" (UID: "ba71f282-bf07-4069-8e39-ce3c3c809db4"). InnerVolumeSpecName "kube-api-access-xvrlz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:17:22.297974 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.297941 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvrlz\" (UniqueName: \"kubernetes.io/projected/ba71f282-bf07-4069-8e39-ce3c3c809db4-kube-api-access-xvrlz\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:22.297974 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.297968 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba71f282-bf07-4069-8e39-ce3c3c809db4-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:22.297974 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:22.297977 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba71f282-bf07-4069-8e39-ce3c3c809db4-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:23.097650 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.097622 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" Jul 13 17:17:23.097650 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.097639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85" event={"ID":"ba71f282-bf07-4069-8e39-ce3c3c809db4","Type":"ContainerDied","Data":"b0b7f160cbeb1e871c7d286f2cf33c55b811c784dc7bd7ec525f46714cd17fcd"} Jul 13 17:17:23.098147 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.097694 2569 scope.go:117] "RemoveContainer" containerID="d52862583460758bc6aad44e5f10fc2b17220b2a3d2412ca0143bbdef512cf10" Jul 13 17:17:23.121482 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.121457 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:17:23.126613 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.126591 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-874fa5-fbbd8577d-wqr85"] Jul 13 17:17:23.720606 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:23.720574 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba71f282-bf07-4069-8e39-ce3c3c809db4" path="/var/lib/kubelet/pods/ba71f282-bf07-4069-8e39-ce3c3c809db4/volumes" Jul 13 17:17:47.875316 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:47.875277 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:17:47.875771 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:47.875534 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" podUID="d7067035-e572-4782-ae72-b658e7ed8e16" containerName="maas-api" containerID="cri-o://ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538" gracePeriod=30 Jul 13 17:17:48.126032 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.125972 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:17:48.216495 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.216460 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca\") pod \"d7067035-e572-4782-ae72-b658e7ed8e16\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " Jul 13 17:17:48.216674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.216500 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") pod \"d7067035-e572-4782-ae72-b658e7ed8e16\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " Jul 13 17:17:48.216674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.216534 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hwfw\" (UniqueName: \"kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw\") pod \"d7067035-e572-4782-ae72-b658e7ed8e16\" (UID: \"d7067035-e572-4782-ae72-b658e7ed8e16\") " Jul 13 17:17:48.216915 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.216889 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7067035-e572-4782-ae72-b658e7ed8e16" (UID: "d7067035-e572-4782-ae72-b658e7ed8e16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:17:48.218686 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.218652 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7067035-e572-4782-ae72-b658e7ed8e16" (UID: "d7067035-e572-4782-ae72-b658e7ed8e16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:17:48.218805 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.218684 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw" (OuterVolumeSpecName: "kube-api-access-6hwfw") pod "d7067035-e572-4782-ae72-b658e7ed8e16" (UID: "d7067035-e572-4782-ae72-b658e7ed8e16"). InnerVolumeSpecName "kube-api-access-6hwfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:17:48.231333 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.231308 2569 generic.go:356] "Generic (PLEG): container finished" podID="d7067035-e572-4782-ae72-b658e7ed8e16" containerID="ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538" exitCode=0 Jul 13 17:17:48.231462 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.231351 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" Jul 13 17:17:48.231462 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.231391 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" event={"ID":"d7067035-e572-4782-ae72-b658e7ed8e16","Type":"ContainerDied","Data":"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538"} Jul 13 17:17:48.231462 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.231440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p" event={"ID":"d7067035-e572-4782-ae72-b658e7ed8e16","Type":"ContainerDied","Data":"d028a1d57ace01a1eddd1f05689dcb888adc9e8177758b68c93195d2c089583a"} Jul 13 17:17:48.231462 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.231462 2569 scope.go:117] "RemoveContainer" containerID="ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538" Jul 13 17:17:48.240390 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.240372 2569 scope.go:117] "RemoveContainer" containerID="ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538" Jul 13 17:17:48.240633 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:17:48.240616 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538\": container with ID starting with ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538 not found: ID does not exist" containerID="ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538" Jul 13 17:17:48.240688 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.240642 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538"} err="failed to get container status \"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538\": rpc error: code = NotFound desc = could not find container \"ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538\": container with ID starting with ba8e20bd9c7fa77e615c0052c7c899b94b43d478a9d01b07683bcaecec3bb538 not found: ID does not exist" Jul 13 17:17:48.255723 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.255702 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:17:48.267695 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.267672 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0c5523-74bd8b7b6d-qfr2p"] Jul 13 17:17:48.317399 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.317379 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7067035-e572-4782-ae72-b658e7ed8e16-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:48.317399 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.317400 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7067035-e572-4782-ae72-b658e7ed8e16-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:48.317551 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:48.317410 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hwfw\" (UniqueName: \"kubernetes.io/projected/d7067035-e572-4782-ae72-b658e7ed8e16-kube-api-access-6hwfw\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:17:49.719937 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:17:49.719895 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7067035-e572-4782-ae72-b658e7ed8e16" path="/var/lib/kubelet/pods/d7067035-e572-4782-ae72-b658e7ed8e16/volumes" Jul 13 17:18:07.492961 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.492926 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493346 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba71f282-bf07-4069-8e39-ce3c3c809db4" containerName="maas-api" Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493359 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba71f282-bf07-4069-8e39-ce3c3c809db4" containerName="maas-api" Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493367 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7067035-e572-4782-ae72-b658e7ed8e16" containerName="maas-api" Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493373 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7067035-e572-4782-ae72-b658e7ed8e16" containerName="maas-api" Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493433 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba71f282-bf07-4069-8e39-ce3c3c809db4" containerName="maas-api" Jul 13 17:18:07.493460 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.493440 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7067035-e572-4782-ae72-b658e7ed8e16" containerName="maas-api" Jul 13 17:18:07.496846 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.496830 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.499381 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.499358 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2386aa-serving-cert\"" Jul 13 17:18:07.506323 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.506297 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:07.582729 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.582701 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.582880 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.582800 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58g9\" (UniqueName: \"kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.582880 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.582821 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.683518 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.683484 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58g9\" (UniqueName: \"kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.683705 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.683523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.683705 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.683581 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.684328 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.684309 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.685925 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.685905 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.699674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.699651 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58g9\" (UniqueName: \"kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9\") pod \"maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.808930 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.808836 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:07.950828 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.950800 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:07.952702 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:18:07.952678 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222c1e1f_dcfd_4630_8c16_904177f17ac9.slice/crio-e537e64dfd85be1a67268231749aa2894b8e7f6a947edc9d6f64043ef4d0b0e3 WatchSource:0}: Error finding container e537e64dfd85be1a67268231749aa2894b8e7f6a947edc9d6f64043ef4d0b0e3: Status 404 returned error can't find the container with id e537e64dfd85be1a67268231749aa2894b8e7f6a947edc9d6f64043ef4d0b0e3 Jul 13 17:18:07.954048 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:07.954015 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:18:08.336394 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:08.336358 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" event={"ID":"222c1e1f-dcfd-4630-8c16-904177f17ac9","Type":"ContainerStarted","Data":"e537e64dfd85be1a67268231749aa2894b8e7f6a947edc9d6f64043ef4d0b0e3"} Jul 13 17:18:09.343355 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:09.343320 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" event={"ID":"222c1e1f-dcfd-4630-8c16-904177f17ac9","Type":"ContainerStarted","Data":"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1"} Jul 13 17:18:09.343740 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:09.343427 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:09.362674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:09.362621 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" podStartSLOduration=1.725014991 podStartE2EDuration="2.362608654s" totalImagesPullingTime="637.593663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:09.360013677 +0000 UTC m=+2304.150101610" watchObservedRunningTime="2026-07-13 17:18:09.362608654 +0000 UTC m=+2304.152696589" Jul 13 17:18:15.354439 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.354404 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:15.580648 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.580616 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:18:15.584373 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.584357 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.587294 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.587272 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9c55b0-serving-cert\"" Jul 13 17:18:15.612285 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.612229 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:18:15.758848 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.758818 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.759032 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.758863 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w4pl\" (UniqueName: \"kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.759032 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.758938 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.859836 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.859806 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w4pl\" (UniqueName: \"kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.860001 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.859848 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.860001 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.859946 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.860108 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:18:15.860070 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-serving-cert: secret "maas-api-e2e-api-b-9c55b0-serving-cert" not found Jul 13 17:18:15.860152 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:18:15.860141 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls podName:89db86f6-3911-4ec7-bb56-4f9295cee8ff nodeName:}" failed. No retries permitted until 2026-07-13 17:18:16.360112563 +0000 UTC m=+2311.150200475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls") pod "maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" (UID: "89db86f6-3911-4ec7-bb56-4f9295cee8ff") : secret "maas-api-e2e-api-b-9c55b0-serving-cert" not found Jul 13 17:18:15.860480 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.860459 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:15.872871 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:15.872804 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w4pl\" (UniqueName: \"kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:16.365003 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:16.364971 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:16.365442 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:18:16.365097 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-serving-cert: secret "maas-api-e2e-api-b-9c55b0-serving-cert" not found Jul 13 17:18:16.365442 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:18:16.365156 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls podName:89db86f6-3911-4ec7-bb56-4f9295cee8ff nodeName:}" failed. No retries permitted until 2026-07-13 17:18:17.36514377 +0000 UTC m=+2312.155231683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls") pod "maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" (UID: "89db86f6-3911-4ec7-bb56-4f9295cee8ff") : secret "maas-api-e2e-api-b-9c55b0-serving-cert" not found Jul 13 17:18:17.373684 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:17.373641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:17.376266 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:17.376240 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:17.394856 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:17.394830 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:17.532861 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:17.532834 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:18:17.534783 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:18:17.534755 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89db86f6_3911_4ec7_bb56_4f9295cee8ff.slice/crio-10144e54984e49283297a1487a6e47e7cf73ca9d9a678b89ab43e448c9cc7806 WatchSource:0}: Error finding container 10144e54984e49283297a1487a6e47e7cf73ca9d9a678b89ab43e448c9cc7806: Status 404 returned error can't find the container with id 10144e54984e49283297a1487a6e47e7cf73ca9d9a678b89ab43e448c9cc7806 Jul 13 17:18:18.391253 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:18.391153 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" event={"ID":"89db86f6-3911-4ec7-bb56-4f9295cee8ff","Type":"ContainerStarted","Data":"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8"} Jul 13 17:18:18.391253 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:18.391211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" event={"ID":"89db86f6-3911-4ec7-bb56-4f9295cee8ff","Type":"ContainerStarted","Data":"10144e54984e49283297a1487a6e47e7cf73ca9d9a678b89ab43e448c9cc7806"} Jul 13 17:18:18.391253 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:18.391241 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:18.411085 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:18.411020 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" podStartSLOduration=3.050217958 podStartE2EDuration="3.411006451s" totalImagesPullingTime="360.788493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:18.409279532 +0000 UTC m=+2313.199367467" watchObservedRunningTime="2026-07-13 17:18:18.411006451 +0000 UTC m=+2313.201094384" Jul 13 17:18:24.401376 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:24.401343 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:18:39.032099 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.032007 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:39.032602 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.032266 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" podUID="222c1e1f-dcfd-4630-8c16-904177f17ac9" containerName="maas-api" containerID="cri-o://b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1" gracePeriod=30 Jul 13 17:18:39.292736 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.292682 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:39.372804 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.372768 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m58g9\" (UniqueName: \"kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9\") pod \"222c1e1f-dcfd-4630-8c16-904177f17ac9\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " Jul 13 17:18:39.372956 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.372827 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls\") pod \"222c1e1f-dcfd-4630-8c16-904177f17ac9\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " Jul 13 17:18:39.372956 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.372859 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca\") pod \"222c1e1f-dcfd-4630-8c16-904177f17ac9\" (UID: \"222c1e1f-dcfd-4630-8c16-904177f17ac9\") " Jul 13 17:18:39.373300 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.373273 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "222c1e1f-dcfd-4630-8c16-904177f17ac9" (UID: "222c1e1f-dcfd-4630-8c16-904177f17ac9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:18:39.374786 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.374763 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "222c1e1f-dcfd-4630-8c16-904177f17ac9" (UID: "222c1e1f-dcfd-4630-8c16-904177f17ac9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:18:39.374845 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.374818 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9" (OuterVolumeSpecName: "kube-api-access-m58g9") pod "222c1e1f-dcfd-4630-8c16-904177f17ac9" (UID: "222c1e1f-dcfd-4630-8c16-904177f17ac9"). InnerVolumeSpecName "kube-api-access-m58g9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:18:39.473654 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.473604 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m58g9\" (UniqueName: \"kubernetes.io/projected/222c1e1f-dcfd-4630-8c16-904177f17ac9-kube-api-access-m58g9\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:18:39.473654 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.473648 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222c1e1f-dcfd-4630-8c16-904177f17ac9-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:18:39.473847 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.473668 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222c1e1f-dcfd-4630-8c16-904177f17ac9-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:18:39.503969 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.503938 2569 generic.go:356] "Generic (PLEG): container finished" podID="222c1e1f-dcfd-4630-8c16-904177f17ac9" containerID="b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1" exitCode=0 Jul 13 17:18:39.504137 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.503982 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" Jul 13 17:18:39.504137 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.503991 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" event={"ID":"222c1e1f-dcfd-4630-8c16-904177f17ac9","Type":"ContainerDied","Data":"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1"} Jul 13 17:18:39.504137 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.504072 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl" event={"ID":"222c1e1f-dcfd-4630-8c16-904177f17ac9","Type":"ContainerDied","Data":"e537e64dfd85be1a67268231749aa2894b8e7f6a947edc9d6f64043ef4d0b0e3"} Jul 13 17:18:39.504137 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.504103 2569 scope.go:117] "RemoveContainer" containerID="b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1" Jul 13 17:18:39.513613 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.513587 2569 scope.go:117] "RemoveContainer" containerID="b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1" Jul 13 17:18:39.513883 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:18:39.513862 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1\": container with ID starting with b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1 not found: ID does not exist" containerID="b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1" Jul 13 17:18:39.513957 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.513891 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1"} err="failed to get container status \"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1\": rpc error: code = NotFound desc = could not find container \"b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1\": container with ID starting with b9a9528cd17ff6e047f079903293bfdeb56de3f4001b4998a1ea12f8caf17de1 not found: ID does not exist" Jul 13 17:18:39.526954 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.526931 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:39.531488 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.531463 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2386aa-6d6958bd5-htgdl"] Jul 13 17:18:39.719892 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:18:39.719848 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222c1e1f-dcfd-4630-8c16-904177f17ac9" path="/var/lib/kubelet/pods/222c1e1f-dcfd-4630-8c16-904177f17ac9/volumes" Jul 13 17:19:04.751548 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:04.751515 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:19:04.752018 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:04.751728 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" podUID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" containerName="maas-api" containerID="cri-o://8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8" gracePeriod=30 Jul 13 17:19:04.990192 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:04.990168 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:19:05.089590 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.089506 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") pod \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " Jul 13 17:19:05.089736 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.089612 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w4pl\" (UniqueName: \"kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl\") pod \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " Jul 13 17:19:05.089736 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.089672 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca\") pod \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\" (UID: \"89db86f6-3911-4ec7-bb56-4f9295cee8ff\") " Jul 13 17:19:05.090073 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.090024 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "89db86f6-3911-4ec7-bb56-4f9295cee8ff" (UID: "89db86f6-3911-4ec7-bb56-4f9295cee8ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:19:05.091565 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.091535 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl" (OuterVolumeSpecName: "kube-api-access-8w4pl") pod "89db86f6-3911-4ec7-bb56-4f9295cee8ff" (UID: "89db86f6-3911-4ec7-bb56-4f9295cee8ff"). InnerVolumeSpecName "kube-api-access-8w4pl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:19:05.091665 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.091574 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "89db86f6-3911-4ec7-bb56-4f9295cee8ff" (UID: "89db86f6-3911-4ec7-bb56-4f9295cee8ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:19:05.190471 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.190442 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89db86f6-3911-4ec7-bb56-4f9295cee8ff-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:19:05.190471 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.190466 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w4pl\" (UniqueName: \"kubernetes.io/projected/89db86f6-3911-4ec7-bb56-4f9295cee8ff-kube-api-access-8w4pl\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:19:05.190471 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.190476 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89db86f6-3911-4ec7-bb56-4f9295cee8ff-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:19:05.651674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.651634 2569 generic.go:356] "Generic (PLEG): container finished" podID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" containerID="8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8" exitCode=0 Jul 13 17:19:05.651848 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.651689 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" Jul 13 17:19:05.651848 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.651705 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" event={"ID":"89db86f6-3911-4ec7-bb56-4f9295cee8ff","Type":"ContainerDied","Data":"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8"} Jul 13 17:19:05.651848 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.651782 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd" event={"ID":"89db86f6-3911-4ec7-bb56-4f9295cee8ff","Type":"ContainerDied","Data":"10144e54984e49283297a1487a6e47e7cf73ca9d9a678b89ab43e448c9cc7806"} Jul 13 17:19:05.651848 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.651810 2569 scope.go:117] "RemoveContainer" containerID="8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8" Jul 13 17:19:05.661124 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.661106 2569 scope.go:117] "RemoveContainer" containerID="8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8" Jul 13 17:19:05.661380 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:19:05.661361 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8\": container with ID starting with 8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8 not found: ID does not exist" containerID="8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8" Jul 13 17:19:05.661438 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.661388 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8"} err="failed to get container status \"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8\": rpc error: code = NotFound desc = could not find container \"8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8\": container with ID starting with 8f86202305bf9c488fbdfae38e7d4139972cf736a7a93832b7bc1f27372a4de8 not found: ID does not exist" Jul 13 17:19:05.673826 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.673799 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:19:05.679280 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.679257 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9c55b0-5b896b496f-vh6xd"] Jul 13 17:19:05.721021 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:05.720997 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" path="/var/lib/kubelet/pods/89db86f6-3911-4ec7-bb56-4f9295cee8ff/volumes" Jul 13 17:19:23.828060 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.827933 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:19:23.828531 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828459 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" containerName="maas-api" Jul 13 17:19:23.828531 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828479 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" containerName="maas-api" Jul 13 17:19:23.828531 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828510 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222c1e1f-dcfd-4630-8c16-904177f17ac9" containerName="maas-api" Jul 13 17:19:23.828531 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828518 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="222c1e1f-dcfd-4630-8c16-904177f17ac9" containerName="maas-api" Jul 13 17:19:23.828702 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828615 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="222c1e1f-dcfd-4630-8c16-904177f17ac9" containerName="maas-api" Jul 13 17:19:23.828702 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.828635 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="89db86f6-3911-4ec7-bb56-4f9295cee8ff" containerName="maas-api" Jul 13 17:19:23.831603 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.831580 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:23.834221 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.834192 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-aac012-serving-cert\"" Jul 13 17:19:23.847147 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.847121 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:19:23.953857 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.953827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:23.954013 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.953895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:23.954013 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:23.953938 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2thwm\" (UniqueName: \"kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.054816 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.054779 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.055011 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.054842 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.055011 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.054878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2thwm\" (UniqueName: \"kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.055011 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:19:24.054963 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-serving-cert: secret "maas-api-e2e-shared-a-aac012-serving-cert" not found Jul 13 17:19:24.055220 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:19:24.055097 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls podName:76f08d2f-76d8-4dab-86a6-d4aac0b65c75 nodeName:}" failed. No retries permitted until 2026-07-13 17:19:24.555074549 +0000 UTC m=+2379.345162473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls") pod "maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" (UID: "76f08d2f-76d8-4dab-86a6-d4aac0b65c75") : secret "maas-api-e2e-shared-a-aac012-serving-cert" not found Jul 13 17:19:24.055556 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.055538 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.068085 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.068061 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2thwm\" (UniqueName: \"kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.559413 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.559377 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.561825 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.561799 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") pod \"maas-api-e2e-shared-a-aac012-5ff4569549-g54mf\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.743202 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.743168 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:24.886878 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:24.886843 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:19:24.887542 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:19:24.887507 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f08d2f_76d8_4dab_86a6_d4aac0b65c75.slice/crio-d23b8543430f0b78b16c9264f6c3a13d0378616f9de34b9fbc1f629afd65438d WatchSource:0}: Error finding container d23b8543430f0b78b16c9264f6c3a13d0378616f9de34b9fbc1f629afd65438d: Status 404 returned error can't find the container with id d23b8543430f0b78b16c9264f6c3a13d0378616f9de34b9fbc1f629afd65438d Jul 13 17:19:25.756389 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:25.756353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" event={"ID":"76f08d2f-76d8-4dab-86a6-d4aac0b65c75","Type":"ContainerStarted","Data":"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee"} Jul 13 17:19:25.756574 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:25.756404 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" event={"ID":"76f08d2f-76d8-4dab-86a6-d4aac0b65c75","Type":"ContainerStarted","Data":"d23b8543430f0b78b16c9264f6c3a13d0378616f9de34b9fbc1f629afd65438d"} Jul 13 17:19:25.756574 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:25.756434 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:25.776193 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:25.776137 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" podStartSLOduration=2.330911793 podStartE2EDuration="2.776122232s" totalImagesPullingTime="445.210439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:19:25.774056969 +0000 UTC m=+2380.564144896" watchObservedRunningTime="2026-07-13 17:19:25.776122232 +0000 UTC m=+2380.566210165" Jul 13 17:19:31.767456 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:31.767422 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:19:31.895298 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:31.895264 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:19:31.899030 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:31.899011 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:31.901677 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:31.901653 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bd468d-serving-cert\"" Jul 13 17:19:31.911328 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:31.911291 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:19:32.029277 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.029183 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.029277 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.029237 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhjb9\" (UniqueName: \"kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.029472 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.029312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.130437 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.130404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.130605 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.130443 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhjb9\" (UniqueName: \"kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.130605 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.130471 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.130605 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:19:32.130557 2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-serving-cert: secret "maas-api-e2e-shared-b-bd468d-serving-cert" not found Jul 13 17:19:32.130711 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:19:32.130624 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls podName:6035855e-b8ee-45b3-a364-e1cb88da47d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:19:32.630608704 +0000 UTC m=+2387.420696616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls") pod "maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" (UID: "6035855e-b8ee-45b3-a364-e1cb88da47d9") : secret "maas-api-e2e-shared-b-bd468d-serving-cert" not found Jul 13 17:19:32.131133 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.131116 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.140656 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.140628 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhjb9\" (UniqueName: \"kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.636122 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.636088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.638489 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.638463 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") pod \"maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.809495 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.809457 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:32.953470 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:32.953443 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:19:32.955543 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:19:32.955503 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6035855e_b8ee_45b3_a364_e1cb88da47d9.slice/crio-6beea18abb9056417384ce2598ba42dafff6a0f6f6e3307390b116bf0eedc77d WatchSource:0}: Error finding container 6beea18abb9056417384ce2598ba42dafff6a0f6f6e3307390b116bf0eedc77d: Status 404 returned error can't find the container with id 6beea18abb9056417384ce2598ba42dafff6a0f6f6e3307390b116bf0eedc77d Jul 13 17:19:33.799259 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:33.799165 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" event={"ID":"6035855e-b8ee-45b3-a364-e1cb88da47d9","Type":"ContainerStarted","Data":"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426"} Jul 13 17:19:33.799259 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:33.799220 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" event={"ID":"6035855e-b8ee-45b3-a364-e1cb88da47d9","Type":"ContainerStarted","Data":"6beea18abb9056417384ce2598ba42dafff6a0f6f6e3307390b116bf0eedc77d"} Jul 13 17:19:33.799502 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:33.799263 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:39.810623 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:39.810593 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:19:39.833238 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:39.833183 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" podStartSLOduration=8.325013573 podStartE2EDuration="8.833167822s" totalImagesPullingTime="508.154249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:19:33.818413124 +0000 UTC m=+2388.608501058" watchObservedRunningTime="2026-07-13 17:19:39.833167822 +0000 UTC m=+2394.623255755" Jul 13 17:19:45.910080 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:45.910052 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:19:45.913815 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:45.913792 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:19:45.914977 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:45.914958 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:19:45.918497 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:19:45.918480 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:24:45.945348 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:24:45.945275 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:24:45.949067 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:24:45.949031 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:24:45.950996 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:24:45.950978 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:24:45.954674 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:24:45.954658 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7jxg_5d19c188-a7a4-415e-b9ae-2446e3399c85/ovn-acl-logging/0.log" Jul 13 17:25:28.386988 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.386952 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:25:28.387716 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.387229 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" podUID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" containerName="maas-api" containerID="cri-o://190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee" gracePeriod=30 Jul 13 17:25:28.637649 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.637588 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:25:28.653189 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.653159 2569 generic.go:356] "Generic (PLEG): container finished" podID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" containerID="190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee" exitCode=0 Jul 13 17:25:28.653303 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.653201 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" Jul 13 17:25:28.653303 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.653237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" event={"ID":"76f08d2f-76d8-4dab-86a6-d4aac0b65c75","Type":"ContainerDied","Data":"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee"} Jul 13 17:25:28.653303 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.653290 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf" event={"ID":"76f08d2f-76d8-4dab-86a6-d4aac0b65c75","Type":"ContainerDied","Data":"d23b8543430f0b78b16c9264f6c3a13d0378616f9de34b9fbc1f629afd65438d"} Jul 13 17:25:28.653412 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.653314 2569 scope.go:117] "RemoveContainer" containerID="190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee" Jul 13 17:25:28.663604 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.663582 2569 scope.go:117] "RemoveContainer" containerID="190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee" Jul 13 17:25:28.663878 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:25:28.663858 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee\": container with ID starting with 190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee not found: ID does not exist" containerID="190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee" Jul 13 17:25:28.663949 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.663888 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee"} err="failed to get container status \"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee\": rpc error: code = NotFound desc = could not find container \"190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee\": container with ID starting with 190a5cf2ba76adff54260cb8d4df90586648fccb7762405a5657453fa5173bee not found: ID does not exist" Jul 13 17:25:28.681242 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.681216 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca\") pod \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " Jul 13 17:25:28.681358 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.681277 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2thwm\" (UniqueName: \"kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm\") pod \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " Jul 13 17:25:28.681358 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.681301 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") pod \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\" (UID: \"76f08d2f-76d8-4dab-86a6-d4aac0b65c75\") " Jul 13 17:25:28.681620 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.681588 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76f08d2f-76d8-4dab-86a6-d4aac0b65c75" (UID: "76f08d2f-76d8-4dab-86a6-d4aac0b65c75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:25:28.683299 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.683278 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76f08d2f-76d8-4dab-86a6-d4aac0b65c75" (UID: "76f08d2f-76d8-4dab-86a6-d4aac0b65c75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:25:28.683397 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.683370 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm" (OuterVolumeSpecName: "kube-api-access-2thwm") pod "76f08d2f-76d8-4dab-86a6-d4aac0b65c75" (UID: "76f08d2f-76d8-4dab-86a6-d4aac0b65c75"). InnerVolumeSpecName "kube-api-access-2thwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:28.782516 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.782490 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2thwm\" (UniqueName: \"kubernetes.io/projected/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-kube-api-access-2thwm\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:28.782516 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.782513 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:28.782677 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.782530 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76f08d2f-76d8-4dab-86a6-d4aac0b65c75-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:28.976278 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.976244 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:25:28.979977 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:28.979950 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-aac012-5ff4569549-g54mf"] Jul 13 17:25:29.719768 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:29.719734 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" path="/var/lib/kubelet/pods/76f08d2f-76d8-4dab-86a6-d4aac0b65c75/volumes" Jul 13 17:25:54.543853 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:54.543819 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:25:54.544451 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:54.544030 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerName="maas-api" containerID="cri-o://9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426" gracePeriod=30 Jul 13 17:25:54.805069 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:54.804962 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.58:8443/health\": dial tcp 10.132.0.58:8443: connect: connection refused" Jul 13 17:25:54.994316 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:54.994293 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:25:55.097791 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.097702 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") pod \"6035855e-b8ee-45b3-a364-e1cb88da47d9\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " Jul 13 17:25:55.097791 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.097751 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca\") pod \"6035855e-b8ee-45b3-a364-e1cb88da47d9\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " Jul 13 17:25:55.097791 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.097773 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhjb9\" (UniqueName: \"kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9\") pod \"6035855e-b8ee-45b3-a364-e1cb88da47d9\" (UID: \"6035855e-b8ee-45b3-a364-e1cb88da47d9\") " Jul 13 17:25:55.098135 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.098087 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6035855e-b8ee-45b3-a364-e1cb88da47d9" (UID: "6035855e-b8ee-45b3-a364-e1cb88da47d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:25:55.099892 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.099864 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6035855e-b8ee-45b3-a364-e1cb88da47d9" (UID: "6035855e-b8ee-45b3-a364-e1cb88da47d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:25:55.100005 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.099920 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9" (OuterVolumeSpecName: "kube-api-access-xhjb9") pod "6035855e-b8ee-45b3-a364-e1cb88da47d9" (UID: "6035855e-b8ee-45b3-a364-e1cb88da47d9"). InnerVolumeSpecName "kube-api-access-xhjb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:55.198866 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.198832 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6035855e-b8ee-45b3-a364-e1cb88da47d9-maas-api-tls\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:55.198866 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.198860 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6035855e-b8ee-45b3-a364-e1cb88da47d9-openshift-service-ca\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:55.199069 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.198874 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhjb9\" (UniqueName: \"kubernetes.io/projected/6035855e-b8ee-45b3-a364-e1cb88da47d9-kube-api-access-xhjb9\") on node \"ip-10-0-141-178.ec2.internal\" DevicePath \"\"" Jul 13 17:25:55.800719 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.800687 2569 generic.go:356] "Generic (PLEG): container finished" podID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerID="9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426" exitCode=0 Jul 13 17:25:55.801115 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.800734 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" Jul 13 17:25:55.801115 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.800808 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" event={"ID":"6035855e-b8ee-45b3-a364-e1cb88da47d9","Type":"ContainerDied","Data":"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426"} Jul 13 17:25:55.801115 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.800863 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v" event={"ID":"6035855e-b8ee-45b3-a364-e1cb88da47d9","Type":"ContainerDied","Data":"6beea18abb9056417384ce2598ba42dafff6a0f6f6e3307390b116bf0eedc77d"} Jul 13 17:25:55.801115 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.800885 2569 scope.go:117] "RemoveContainer" containerID="9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426" Jul 13 17:25:55.810067 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.810006 2569 scope.go:117] "RemoveContainer" containerID="9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426" Jul 13 17:25:55.810454 ip-10-0-141-178 kubenswrapper[2569]: E0713 17:25:55.810424 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426\": container with ID starting with 9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426 not found: ID does not exist" containerID="9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426" Jul 13 17:25:55.810585 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.810473 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426"} err="failed to get container status \"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426\": rpc error: code = NotFound desc = could not find container \"9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426\": container with ID starting with 9e251699d675af725357f0851d6c5d63cb0afd231d260d15a83a0383d0d45426 not found: ID does not exist" Jul 13 17:25:55.819394 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.819371 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:25:55.823591 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:55.823568 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bd468d-fc74bc57b-tsc7v"] Jul 13 17:25:57.720150 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:25:57.720115 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" path="/var/lib/kubelet/pods/6035855e-b8ee-45b3-a364-e1cb88da47d9/volumes" Jul 13 17:26:15.766618 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:15.766532 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69cbf74464-rtf77_a4911f1f-b410-400d-b93f-5a83774587c8/manager/0.log" Jul 13 17:26:16.416645 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:16.416618 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-xk7ds_df850945-13c1-43e6-af5b-953b50c9f720/copy-manifests/0.log" Jul 13 17:26:16.425446 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:16.425426 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-xk7ds_df850945-13c1-43e6-af5b-953b50c9f720/manager/0.log" Jul 13 17:26:16.793454 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:16.793378 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6ddc58579c-4n76t_1838a6a2-a23b-4059-8f96-3bff84ca679b/maas-api/0.log" Jul 13 17:26:18.547101 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:18.547072 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4zddt_f5d7f546-740d-4791-b7a7-a60f8f452e21/postgres/0.log" Jul 13 17:26:19.348291 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.348257 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/util/0.log" Jul 13 17:26:19.356959 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.356929 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/pull/0.log" Jul 13 17:26:19.364389 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.364365 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/extract/0.log" Jul 13 17:26:19.476692 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.476661 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/util/0.log" Jul 13 17:26:19.483433 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.483407 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/pull/0.log" Jul 13 17:26:19.490852 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.490835 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/extract/0.log" Jul 13 17:26:19.608268 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.608202 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/extract/0.log" Jul 13 17:26:19.615826 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.615799 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/util/0.log" Jul 13 17:26:19.623331 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.623307 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/pull/0.log" Jul 13 17:26:19.740445 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.740420 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/util/0.log" Jul 13 17:26:19.747665 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.747646 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/pull/0.log" Jul 13 17:26:19.754543 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:19.754514 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/extract/0.log" Jul 13 17:26:21.260643 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:21.260607 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cgp86_ded7ff20-866f-43ec-9815-3a3712ffea91/discovery/0.log" Jul 13 17:26:21.858679 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:21.858628 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-52brd_86dc2ce4-525b-4449-8dbf-a2a7b861c540/payload-processing/0.log" Jul 13 17:26:21.972789 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:21.972756 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7657f6f484-wcq5g_1d1f00f5-f46d-4962-8bfb-c49decad5994/router/0.log" Jul 13 17:26:22.544371 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:22.544339 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ht6w5_b4d6ea6e-8bfc-44c1-9d4f-c277612870b2/main/0.log" Jul 13 17:26:22.657262 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:22.657234 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbj529d_1c95ad49-b766-405d-97c1-cb271df013dd/main/0.log" Jul 13 17:26:30.080699 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:30.080671 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-grd9b_81257490-18b6-48fa-8601-eef4639687e1/global-pull-secret-syncer/0.log" Jul 13 17:26:30.238352 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:30.238325 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nbzzc_b7e58ff0-43a6-464f-a81a-7a1a3d8a8b9f/konnectivity-agent/0.log" Jul 13 17:26:30.284065 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:30.284014 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-178.ec2.internal_1c9b46fdf787e2321bc5c8391e702436/haproxy/0.log" Jul 13 17:26:34.827967 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.827929 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/extract/0.log" Jul 13 17:26:34.851417 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.851395 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/util/0.log" Jul 13 17:26:34.873056 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.873020 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n7sqc_928158e2-5a02-4549-beb5-bfb28e65a1a4/pull/0.log" Jul 13 17:26:34.907023 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.907006 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/extract/0.log" Jul 13 17:26:34.935165 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.935103 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/util/0.log" Jul 13 17:26:34.959545 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.959526 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ppqbj_af4728ae-9937-4dae-a2e7-14e89fd79f0d/pull/0.log" Jul 13 17:26:34.992646 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:34.992624 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/extract/0.log" Jul 13 17:26:35.018056 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:35.018018 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/util/0.log" Jul 13 17:26:35.041678 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:35.041657 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qhwrr_78d4ea9b-2822-48ee-a769-55831ff12212/pull/0.log" Jul 13 17:26:35.071684 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:35.071663 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/extract/0.log" Jul 13 17:26:35.093206 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:35.093187 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/util/0.log" Jul 13 17:26:35.120094 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:35.120033 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ntfp5_80a086ea-fed8-4596-a55d-443d3b1381ca/pull/0.log" Jul 13 17:26:37.937292 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:37.937261 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-fgszt_7425c4b6-381c-445f-828e-98d197d6d0c6/cluster-monitoring-operator/0.log" Jul 13 17:26:38.171534 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:38.171505 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-529j7_b4b54637-6577-497b-9330-c3be6a3e37a1/node-exporter/0.log" Jul 13 17:26:38.200666 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:38.200592 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-529j7_b4b54637-6577-497b-9330-c3be6a3e37a1/kube-rbac-proxy/0.log" Jul 13 17:26:38.226621 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:38.226600 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-529j7_b4b54637-6577-497b-9330-c3be6a3e37a1/init-textfile/0.log" Jul 13 17:26:40.236823 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.236789 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-wjqdg_816d33a9-20d0-440b-a9f3-914311397007/networking-console-plugin/0.log" Jul 13 17:26:40.568010 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.567926 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk"] Jul 13 17:26:40.568294 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568281 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" containerName="maas-api" Jul 13 17:26:40.568343 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568296 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" containerName="maas-api" Jul 13 17:26:40.568343 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568314 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerName="maas-api" Jul 13 17:26:40.568343 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568320 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerName="maas-api" Jul 13 17:26:40.568437 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568373 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f08d2f-76d8-4dab-86a6-d4aac0b65c75" containerName="maas-api" Jul 13 17:26:40.568437 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.568382 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6035855e-b8ee-45b3-a364-e1cb88da47d9" containerName="maas-api" Jul 13 17:26:40.571709 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.571688 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.574562 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.574542 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqxjc\"/\"kube-root-ca.crt\"" Jul 13 17:26:40.575579 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.575559 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mqxjc\"/\"default-dockercfg-8zq4g\"" Jul 13 17:26:40.575671 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.575559 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqxjc\"/\"openshift-service-ca.crt\"" Jul 13 17:26:40.583370 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.583348 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk"] Jul 13 17:26:40.671425 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.671396 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-lib-modules\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.671609 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.671434 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-podres\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.671609 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.671503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpkn\" (UniqueName: \"kubernetes.io/projected/424146d8-6f1d-4920-92c6-039746cc013d-kube-api-access-qhpkn\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.671609 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.671561 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-sys\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.671609 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.671599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-proc\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772586 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772553 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-proc\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772747 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772617 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-lib-modules\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772747 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772643 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-podres\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772747 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772682 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-proc\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772747 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772695 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpkn\" (UniqueName: \"kubernetes.io/projected/424146d8-6f1d-4920-92c6-039746cc013d-kube-api-access-qhpkn\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772747 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772728 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-sys\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772972 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-sys\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772972 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772810 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-podres\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.772972 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.772813 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/424146d8-6f1d-4920-92c6-039746cc013d-lib-modules\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.781800 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.781779 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpkn\" (UniqueName: \"kubernetes.io/projected/424146d8-6f1d-4920-92c6-039746cc013d-kube-api-access-qhpkn\") pod \"perf-node-gather-daemonset-2hbjk\" (UID: \"424146d8-6f1d-4920-92c6-039746cc013d\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.881783 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.881701 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:40.903387 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.903354 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/1.log" Jul 13 17:26:40.908666 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:40.908644 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kmkjt_d5cb9f57-2e10-4550-b60f-6d7956a460ea/console-operator/2.log" Jul 13 17:26:41.019364 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:41.019341 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk"] Jul 13 17:26:41.020842 ip-10-0-141-178 kubenswrapper[2569]: W0713 17:26:41.020816 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod424146d8_6f1d_4920_92c6_039746cc013d.slice/crio-3a9235a08dc2756b366e6eb0fbb5f7655ce4613ee64fbbead4cb857f2ebf2779 WatchSource:0}: Error finding container 3a9235a08dc2756b366e6eb0fbb5f7655ce4613ee64fbbead4cb857f2ebf2779: Status 404 returned error can't find the container with id 3a9235a08dc2756b366e6eb0fbb5f7655ce4613ee64fbbead4cb857f2ebf2779 Jul 13 17:26:41.027755 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:41.027659 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" event={"ID":"424146d8-6f1d-4920-92c6-039746cc013d","Type":"ContainerStarted","Data":"3a9235a08dc2756b366e6eb0fbb5f7655ce4613ee64fbbead4cb857f2ebf2779"} Jul 13 17:26:41.468994 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:41.468964 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-bgv2v_1cb7c776-b672-4346-8cb7-f88c8fb2a34e/download-server/0.log" Jul 13 17:26:42.034616 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:42.034568 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" event={"ID":"424146d8-6f1d-4920-92c6-039746cc013d","Type":"ContainerStarted","Data":"f962418e5a95d2ad1ead2c0f3c912aa710cf96ea45ec3674e2c00e928706e364"} Jul 13 17:26:42.034792 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:42.034682 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:42.051145 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:42.051095 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" podStartSLOduration=2.051080549 podStartE2EDuration="2.051080549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:42.049983771 +0000 UTC m=+2816.840071708" watchObservedRunningTime="2026-07-13 17:26:42.051080549 +0000 UTC m=+2816.841168483" Jul 13 17:26:42.056391 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:42.056372 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-4tqkf_67d6774c-170b-4704-8f96-3eab4de7fbf1/volume-data-source-validator/0.log" Jul 13 17:26:43.208063 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:43.208020 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zftr5_267f22ff-d752-4298-923c-5baa9e0ad6e6/dns/0.log" Jul 13 17:26:43.232448 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:43.232427 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zftr5_267f22ff-d752-4298-923c-5baa9e0ad6e6/kube-rbac-proxy/0.log" Jul 13 17:26:43.260439 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:43.260415 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2ppc_4cc7aca3-ab5d-4c42-af22-45450b82be31/dns-node-resolver/0.log" Jul 13 17:26:43.910192 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:43.910162 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7m9tz_c5b7a469-647c-4afe-91a5-e11edfb96827/node-ca/0.log" Jul 13 17:26:45.229428 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:45.229397 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cgp86_ded7ff20-866f-43ec-9815-3a3712ffea91/discovery/0.log" Jul 13 17:26:45.450759 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:45.450699 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-52brd_86dc2ce4-525b-4449-8dbf-a2a7b861c540/payload-processing/0.log" Jul 13 17:26:45.474913 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:45.474886 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7657f6f484-wcq5g_1d1f00f5-f46d-4962-8bfb-c49decad5994/router/0.log" Jul 13 17:26:46.265501 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.265472 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9b2j_25174820-93a8-4044-8154-889410fec806/serve-healthcheck-canary/0.log" Jul 13 17:26:46.750846 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.750810 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7rwks_5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6/insights-operator/0.log" Jul 13 17:26:46.751572 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.751552 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-7rwks_5eb4dc3e-ec0a-4ef1-b94f-5579b3d374b6/insights-operator/1.log" Jul 13 17:26:46.857160 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.857120 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq2g9_8f1f6b35-4aea-4570-bf35-4f6e46680e83/kube-rbac-proxy/0.log" Jul 13 17:26:46.881120 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.881095 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq2g9_8f1f6b35-4aea-4570-bf35-4f6e46680e83/exporter/0.log" Jul 13 17:26:46.904988 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:46.904964 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq2g9_8f1f6b35-4aea-4570-bf35-4f6e46680e83/extractor/0.log" Jul 13 17:26:48.049771 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:48.049740 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2hbjk" Jul 13 17:26:49.484727 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:49.484640 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-69cbf74464-rtf77_a4911f1f-b410-400d-b93f-5a83774587c8/manager/0.log" Jul 13 17:26:49.877358 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:49.877328 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-xk7ds_df850945-13c1-43e6-af5b-953b50c9f720/manager/0.log" Jul 13 17:26:49.955396 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:49.955372 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-xk7ds_df850945-13c1-43e6-af5b-953b50c9f720/copy-manifests/0.log" Jul 13 17:26:51.168574 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:51.168542 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cgp86_ded7ff20-866f-43ec-9815-3a3712ffea91/discovery/0.log" Jul 13 17:26:51.293839 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:51.293805 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-52brd_86dc2ce4-525b-4449-8dbf-a2a7b861c540/payload-processing/0.log" Jul 13 17:26:51.307613 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:51.307587 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7657f6f484-wcq5g_1d1f00f5-f46d-4962-8bfb-c49decad5994/router/0.log" Jul 13 17:26:51.361308 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:51.361284 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78cc96c4cb-nwcsm_61ac3113-0bf3-43fc-bcb0-bc617d8210df/manager/0.log" Jul 13 17:26:51.388237 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:51.388217 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bltsj_02dd630a-f3f0-438c-a623-9ee603c74520/openshift-lws-operator/0.log" Jul 13 17:26:52.706053 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:52.706016 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cgp86_ded7ff20-866f-43ec-9815-3a3712ffea91/discovery/0.log" Jul 13 17:26:52.844695 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:52.844534 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-52brd_86dc2ce4-525b-4449-8dbf-a2a7b861c540/payload-processing/0.log" Jul 13 17:26:52.858626 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:52.858599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7657f6f484-wcq5g_1d1f00f5-f46d-4962-8bfb-c49decad5994/router/0.log" Jul 13 17:26:56.457781 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:56.457753 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-rzqdd_3084f010-9683-4a3e-afdb-ab8edae69544/migrator/0.log" Jul 13 17:26:56.480498 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:56.480472 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-rzqdd_3084f010-9683-4a3e-afdb-ab8edae69544/graceful-termination/0.log" Jul 13 17:26:56.846817 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:56.846732 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2n6wl_f6a72c4e-2845-4db9-9cae-ee53a2a8daad/kube-storage-version-migrator-operator/1.log" Jul 13 17:26:56.847509 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:56.847491 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-2n6wl_f6a72c4e-2845-4db9-9cae-ee53a2a8daad/kube-storage-version-migrator-operator/0.log" Jul 13 17:26:57.862894 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:57.862865 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2jlz5_163640c8-2bec-4876-a451-71fd060c9c9f/kube-multus/0.log" Jul 13 17:26:58.136276 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.136198 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/kube-multus-additional-cni-plugins/0.log" Jul 13 17:26:58.159470 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.159444 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/egress-router-binary-copy/0.log" Jul 13 17:26:58.183920 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.183893 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/cni-plugins/0.log" Jul 13 17:26:58.210257 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.210234 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/bond-cni-plugin/0.log" Jul 13 17:26:58.240465 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.240442 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/routeoverride-cni/0.log" Jul 13 17:26:58.266019 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.266002 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/whereabouts-cni-bincopy/0.log" Jul 13 17:26:58.292026 ip-10-0-141-178 kubenswrapper[2569]: I0713 17:26:58.292003 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rw5cq_72abcb4e-bd9b-4846-85c4-132f6a46fabf/whereabouts-cni/0.log"